Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion docs/efcore.md
Original file line number Diff line number Diff line change
Expand Up @@ -926,7 +926,7 @@ translation, and in the final projection EF runs them in memory instead:
| The value set indexer, `set[0]` | Subscripting reads the stored array, which is canonical only for rows this library wrote ([Rows written by other tools](#rows-written-by-other-tools)). PostgreSQL arrays also start at 1, and an index past the end is `NULL` where memory throws. |
| `Values` on a value set | It is the loaded array itself. In a query, use the set's own members, such as `Contains` and `Count`. |
| `Count` and the indexer on a `RangeSet` | Not translated. PostgreSQL has no function for either, and each would be a subquery over `unnest(x)`. |
| `Compare` on `RangeLowerBoundComparer`, `RangeSet.LowerBoundComparer` and any other `IComparer` of ranges or sets | EF Core reads every `Compare(x, y)` as the natural order of `x` and `y`, whatever the comparer. In SQL the lower-bound comparer would compare in the range order. For the range order in SQL, call `x.CompareTo(y)`. |
| `Compare` on `RangeLowerBoundComparer`, `RangeSet.LowerBoundComparer` and any other `IComparer` of ranges or sets | EF Core reads every `Compare(x, y)` as the natural order of `x` and `y`, whatever the comparer. In SQL the lower-bound comparer would compare in the range order. What counts is what you compare, not the type the comparer is declared over: an `IComparer<IRange<T>>`, a non-generic `IComparer` and a comparer of value sets run in memory too. For the range order in SQL, call `x.CompareTo(y)`. |
| `From` and `Parse` on any range or set | They build a value in .NET. Given values your code passes in, EF Core runs them before the query and sends the result as a parameter. |
| `ToInstantRange`, `ToInterval`, `ToLocalDateRange`, `ToDateInterval` and `ToYearMonthRange` (NodaTime package) | They convert in .NET: between NodaTime's `Interval` or `DateInterval` and a range type, or between `LocalDateRange` and `YearMonthRange`. |

Expand Down
4 changes: 3 additions & 1 deletion docs/migration.md
Original file line number Diff line number Diff line change
Expand Up @@ -994,7 +994,9 @@ in SQL in that order. Now it runs in memory only, like the other members under
| `Select(r => comparer.Compare(r.Period, p))`, for a NULL `Period` | throws (`InvalidOperationException`) | `-1`, computed in memory |

`Comparer<DateRange>.Default.Compare` runs in memory too. Once the comparer is a query parameter, a
query can't tell one comparer from another.
query can't tell one comparer from another. So do comparers declared over another type, when what they
compare is a range or a set: an `IComparer<IRange<DateOnly>>`, a non-generic `IComparer` (such as
`Comparer<DateRange>.Create(…)` held as one) and an `IComparer<StringSet>`.

- **Search your queries for `Compare(`.** Each call on a comparer of ranges or sets in a `Where` or
an `OrderBy` now throws, so running the query finds it.
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -3,7 +3,8 @@
namespace CodoMetis.ValueRanges.EntityFrameworkCore.PostgreSQL.Query;

/// <summary>
/// The form an <see cref="IComparer{T}"/>'s <c>Compare</c> over two ranges or two sets takes in a query
/// The form an <see cref="IComparer{T}"/>'s or <see cref="System.Collections.IComparer"/>'s <c>Compare</c>
/// over a range, a range set or a value set takes in a query
/// once <see cref="ValueRangesQueryExpressionInterceptor"/> has seen it: a call no translator knows, so
/// it runs in memory or not at all (#385).
/// </summary>
Expand All @@ -17,7 +18,14 @@ namespace CodoMetis.ValueRanges.EntityFrameworkCore.PostgreSQL.Query;
internal static class RangeComparerCall
{
/// <summary><paramref name="comparer"/>'s own <see cref="IComparer{T}.Compare"/>.</summary>
public static int RunInMemory<TRange>(IComparer<TRange> comparer, TRange x, TRange y) => comparer.Compare(x, y);
public static int RunInMemory<T>(IComparer<T> comparer, T x, T y) => comparer.Compare(x, y);

internal static readonly MethodInfo RunInMemoryMethod = typeof(RangeComparerCall).GetMethod(nameof(RunInMemory))!;
/// <summary><paramref name="comparer"/>'s own <see cref="System.Collections.IComparer.Compare"/>.</summary>
public static int RunInMemory(System.Collections.IComparer comparer, object? x, object? y) => comparer.Compare(x, y);

internal static readonly MethodInfo RunInMemoryMethod =
typeof(RangeComparerCall).GetMethods().Single(method => method is { Name: nameof(RunInMemory), IsGenericMethodDefinition: true });

internal static readonly MethodInfo RunInMemoryNonGenericMethod =
typeof(RangeComparerCall).GetMethod(nameof(RunInMemory), [typeof(System.Collections.IComparer), typeof(object), typeof(object)])!;
}
Original file line number Diff line number Diff line change
@@ -1,4 +1,5 @@
using System.Linq.Expressions;
using System.Reflection;
using CodoMetis.ValueRanges.EntityFrameworkCore.PostgreSQL.Internal;
using Microsoft.EntityFrameworkCore;
using Microsoft.EntityFrameworkCore.Diagnostics;
Expand Down Expand Up @@ -362,8 +363,8 @@ protected override Expression VisitMethodCall(MethodCallExpression node)
}

/// <summary>
/// Turns <c>comparer.Compare(x, y)</c> over two ranges or two sets into
/// <see cref="RangeComparerCall.RunInMemory{TRange}"/>, which EF Core cannot translate: a filter or an
/// Turns <c>comparer.Compare(x, y)</c> over a range, a range set or a value set into
/// <see cref="RangeComparerCall.RunInMemory{T}"/>, which EF Core cannot translate: a filter or an
/// ordering then fails translation, and a final projection runs the comparer in memory (#385).
/// </summary>
/// <remarks>
Expand All @@ -382,6 +383,14 @@ protected override Expression VisitMethodCall(MethodCallExpression node)
/// range or set type is kept out of SQL, <see cref="Comparer{T}.Default"/> included: <c>CompareTo</c>
/// is the call that runs in SQL in the range order.
/// </para>
/// <para>
/// The operands decide, not the type <c>Compare</c> declares. A comparer declared over
/// <see cref="Core.IRange{T}"/>, the non-generic <see cref="System.Collections.IComparer"/> (which
/// <see cref="Comparer{T}.Create"/> is, held as one) and a comparer of value sets each declare a type
/// <see cref="RangeTypeRegistry"/> does not hold, and each still compared in the range order in SQL
/// (#452). So a call is rewritten when either operand, under the conversions to the declared type, is a
/// range, a range set or a value set.
/// </para>
/// </remarks>
private sealed class RangeComparerRewriter : ExpressionVisitor
{
Expand All @@ -392,10 +401,42 @@ protected override Expression VisitMethodCall(MethodCallExpression node)
&& method.ReturnType == typeof(int)
&& method.GetParameters() is [{ ParameterType: var compared }, { ParameterType: var other }]
&& compared == other
&& RangeTypeRegistry.TryGetByClrType(compared, out _, out _)
&& typeof(IComparer<>).MakeGenericType(compared).IsAssignableFrom(comparer.Type)
? Expression.Call(RangeComparerCall.RunInMemoryMethod.MakeGenericMethod(compared), Visit(comparer), Visit(x), Visit(y))
&& (IsRangeOrSet(x) || IsRangeOrSet(y))
&& RunInMemory(comparer.Type, compared) is { } runInMemory
? Expression.Call(runInMemory, Visit(comparer), Visit(x), Visit(y))
: base.VisitMethodCall(node);

private static MethodInfo? RunInMemory(Type comparerType, Type compared)
=> typeof(IComparer<>).MakeGenericType(compared).IsAssignableFrom(comparerType)
? RangeComparerCall.RunInMemoryMethod.MakeGenericMethod(compared)
: compared == typeof(object) && typeof(System.Collections.IComparer).IsAssignableFrom(comparerType)
? RangeComparerCall.RunInMemoryNonGenericMethod
: null;

// The operand and every conversion it passes through on its way to the declared parameter type:
// a column converted to IRange<T>, object or IComparable is still a range.
private static bool IsRangeOrSet(Expression operand)
{
while (!IsRangeOrSetType(operand.Type))
{
if (operand is not UnaryExpression { NodeType: ExpressionType.Convert or ExpressionType.ConvertChecked or ExpressionType.TypeAs } conversion)
return false;

operand = conversion.Operand;
}

return true;
}

// A sealed variant is a range through its union.
private static bool IsRangeOrSetType(Type type)
{
for (var current = Nullable.GetUnderlyingType(type) ?? type; current is not null; current = current.BaseType)
if (RangeTypeRegistry.TryGetByClrType(current, out _, out _) || SetTypeRegistry.TryGetByClrType(current, out _))
return true;

return false;
}
}

private sealed class RangeSetOperatorRewriter : ExpressionVisitor
Expand Down
Original file line number Diff line number Diff line change
@@ -1,4 +1,5 @@
using System.Linq.Expressions;
using CodoMetis.ValueRanges.Core;
using CodoMetis.ValueRanges.EFCore.PostgreSQL.IntegrationTests.Composition;
using Microsoft.EntityFrameworkCore;

Expand All @@ -15,7 +16,10 @@ namespace CodoMetis.ValueRanges.EFCore.PostgreSQL.IntegrationTests;
/// <c>Where(… == 0)</c> selected one row of the three. The comparer is reached directly, through
/// <see cref="IComparer{T}"/>, and through <see cref="RangeSet{TRange,T}.LowerBoundComparer"/>, which a query
/// cannot tell apart once it is a parameter; <see cref="Comparer{T}.Default"/> goes the same way. A NULL
/// column sorts first, as the comparer says.
/// column sorts first, as the comparer says. The operands decide, not the type <c>Compare</c> declares: a
/// comparer over <see cref="IRange{T}"/>, the non-generic <see cref="System.Collections.IComparer"/> and a
/// comparer of value sets each still compared in the range order (#452). By count, <c>{z}</c> is equal to
/// rows 1 and 2, where SQL's <c>=</c> selected none.
/// </remarks>
[TestClass]
public sealed class LowerBoundComparerQueryTests
Expand All @@ -33,6 +37,19 @@ public sealed class LowerBoundComparerQueryTests
null
];

private static readonly StringSet?[] Tags = [StringSet.From(["a"]), StringSet.From(["b"]), StringSet.From(["a", "b"]), StringSet.Empty, null];

private static readonly IComparer<IRange<DateOnly>> OverInterface =
Comparer<IRange<DateOnly>>.Create((x, y) => RangeLowerBoundComparer<DateRange, DateOnly>.Instance.Compare((DateRange?)x, (DateRange?)y));

private static readonly System.Collections.IComparer Created = Comparer<DateRange>.Create(RangeLowerBoundComparer<DateRange, DateOnly>.Instance.Compare);

private static readonly System.Collections.IComparer OwnNonGeneric = new LowerBoundObjectComparer();

private static readonly IComparer<StringSet> ByCount = Comparer<StringSet>.Create((x, y) => (x?.Count ?? -1).CompareTo(y?.Count ?? -1));

private static readonly StringSet OneTag = StringSet.From(["z"]);

[TestMethod]
public async Task Compare_InAProjection_AnswersAsTheComparerDoes()
{
Expand Down Expand Up @@ -60,6 +77,13 @@ async Task Agree(string label, Expression<Func<ComparerRow, int>> compare)
await Agree("IComparer<DateRange>.Compare", r => viaInterface.Compare(r.Period, p));
await Agree("RangeSet.LowerBoundComparer.Compare", r => RangeSet<DateRange, DateOnly>.LowerBoundComparer.Compare(p, r.Period));
await Agree("Comparer<DateRange>.Default.Compare", r => Comparer<DateRange>.Default.Compare(r.Period, p));

// As locals, so each comparer is a query parameter, as in the filter test.
var (overInterface, created, ownNonGeneric, byCount, oneTag) = (OverInterface, Created, OwnNonGeneric, ByCount, OneTag);
await Agree("IComparer<IRange<DateOnly>>.Compare", r => overInterface.Compare(r.Period, p));
await Agree("Comparer<DateRange>.Create as IComparer", r => created.Compare(r.Period, p));
await Agree("own IComparer", r => ownNonGeneric.Compare(r.Period, p));
await Agree("IComparer<StringSet>.Compare", r => byCount.Compare(r.Tags, oneTag));
}

[TestMethod]
Expand All @@ -71,12 +95,24 @@ public async Task Compare_InAFilterOrAnOrdering_FailsTranslation()
IComparer<DateRange> viaInterface = RangeLowerBoundComparer<DateRange, DateOnly>.Instance;
var p = P;

// As locals, so each comparer is a query parameter. EF Core leaves a Compare on a constant comparer
// (a static field) untranslated in any case, and the range order needs a parameter (#452).
var (overInterface, created, ownNonGeneric, byCount, oneTag) = (OverInterface, Created, OwnNonGeneric, ByCount, OneTag);

IQueryable<ComparerRow>[] queries =
[
context.Rows.Where(r => RangeLowerBoundComparer<DateRange, DateOnly>.Instance.Compare(r.Period, p) == 0),
context.Rows.Where(r => viaInterface.Compare(r.Period, p) == 0),
context.Rows.Where(r => RangeSet<DateRange, DateOnly>.LowerBoundComparer.Compare(r.Period, p) <= 0),
context.Rows.OrderBy(r => RangeLowerBoundComparer<DateRange, DateOnly>.Instance.Compare(r.Period, p)),
context.Rows.Where(r => overInterface.Compare(r.Period, p) == 0),
context.Rows.OrderBy(r => overInterface.Compare(r.Period, p)),
context.Rows.Where(r => created.Compare(r.Period, p) == 0),
context.Rows.OrderBy(r => created.Compare(r.Period, p)),
context.Rows.Where(r => ownNonGeneric.Compare(r.Period, p) == 0),
context.Rows.OrderBy(r => ownNonGeneric.Compare(r.Period, p)),
context.Rows.Where(r => byCount.Compare(r.Tags, oneTag) == 0),
context.Rows.OrderBy(r => byCount.Compare(r.Tags, oneTag)),
];

foreach (var query in queries)
Expand All @@ -92,7 +128,7 @@ private static async Task<string> Seed()
var connectionString = await CompositionDatabase.Create("lower_bound_comparer");
await using var seed = new ComparerContext(connectionString);
await seed.Database.EnsureCreatedAsync();
seed.Rows.AddRange(Periods.Select((period, index) => new ComparerRow { Id = index + 1, Period = period }));
seed.Rows.AddRange(Periods.Select((period, index) => new ComparerRow { Id = index + 1, Period = period, Tags = Tags[index] }));
await seed.SaveChangesAsync();
return connectionString;
}
Expand All @@ -102,6 +138,13 @@ public sealed class ComparerRow
public int Id { get; set; }

public DateRange? Period { get; set; }

public StringSet? Tags { get; set; }
}

private sealed class LowerBoundObjectComparer : System.Collections.IComparer
{
public int Compare(object? x, object? y) => RangeLowerBoundComparer<DateRange, DateOnly>.Instance.Compare((DateRange?)x, (DateRange?)y);
}

private sealed class ComparerContext(string connectionString) : DbContext
Expand Down
Original file line number Diff line number Diff line change
@@ -1,3 +1,4 @@
using CodoMetis.ValueRanges.Core;
using Microsoft.EntityFrameworkCore;

namespace CodoMetis.ValueRanges.EFCore.PostgreSQL.Tests;
Expand Down Expand Up @@ -34,8 +35,25 @@ public sealed class ClientSideOnlyTranslationTests
["RangeSet.LowerBoundComparer.Compare", (Func<TestDbContext, IQueryable<Booking>>)(db => db.Bookings.Where(b => RangeSet<DateRange, DateOnly>.LowerBoundComparer.Compare(b.Period, June) > 0))],
["IComparer<RangeSet>.Compare", (Func<TestDbContext, IQueryable<Booking>>)(db => db.Bookings.Where(b => Comparer<RangeSet<DateRange, DateOnly>>.Default.Compare(b.BlockedDays, RangeSet<DateRange, DateOnly>.Empty) == 0))],
["OrderBy(RangeLowerBoundComparer.Compare)", (Func<TestDbContext, IQueryable<Booking>>)(db => db.Bookings.OrderBy(b => RangeLowerBoundComparer<Int32Range, int>.Instance.Compare(b.Seats, Int32Range.Infinite)))],
// The operands decide, not the type Compare declares (#452). Each comparer is a captured local, so a
// query parameter: EF Core leaves Compare on a constant comparer untranslated in any case.
["IComparer<IRange<T>>.Compare", (Func<TestDbContext, IQueryable<Booking>>)(db => { var comparer = OverInterface; return db.Bookings.Where(b => comparer.Compare(b.Period, June) == 0); })],
["OrderBy(IComparer<IRange<T>>.Compare)", (Func<TestDbContext, IQueryable<Booking>>)(db => { var comparer = OverInterface; return db.Bookings.OrderBy(b => comparer.Compare(b.Period, June)); })],
["IComparer.Compare", (Func<TestDbContext, IQueryable<Booking>>)(db => { var comparer = NonGeneric; return db.Bookings.Where(b => comparer.Compare(b.Period, June) == 0); })],
["OrderBy(IComparer.Compare)", (Func<TestDbContext, IQueryable<Booking>>)(db => { var comparer = NonGeneric; return db.Bookings.OrderBy(b => comparer.Compare(b.Period, June)); })],
["IComparer<StringSet>.Compare", (Func<TestDbContext, IQueryable<Booking>>)(db => { var comparer = OverValueSets; return db.Bookings.Where(b => comparer.Compare(b.Tags, OneTag) == 0); })],
["OrderBy(IComparer<StringSet>.Compare)", (Func<TestDbContext, IQueryable<Booking>>)(db => { var comparer = OverValueSets; return db.Bookings.OrderBy(b => comparer.Compare(b.Tags, OneTag)); })],
];

private static readonly IComparer<IRange<DateOnly>> OverInterface =
Comparer<IRange<DateOnly>>.Create((x, y) => RangeLowerBoundComparer<DateRange, DateOnly>.Instance.Compare((DateRange)x, (DateRange)y));

private static readonly System.Collections.IComparer NonGeneric = Comparer<DateRange>.Create(RangeLowerBoundComparer<DateRange, DateOnly>.Instance.Compare);

private static readonly IComparer<StringSet> OverValueSets = Comparer<StringSet>.Create((x, y) => x.Count.CompareTo(y.Count));

private static readonly StringSet OneTag = StringSet.From(["z"]);

private static readonly DateRange June = DateRange.Month(2026, 6);

[TestMethod]
Expand Down
2 changes: 1 addition & 1 deletion test/oracles/claims.json
Original file line number Diff line number Diff line change
Expand Up @@ -583,7 +583,7 @@
"id": "E31-range-client-side-only-operations",
"source": "docs/efcore.md#what-runs-where",
"quote": "**Runs in memory only.** These compute the right answer in memory. In a predicate they fail translation, and in the final projection EF runs them in memory instead:",
"claim": "Length (on every range type but Int32Range and Int64Range), Values(), Values(step), Split, Clamp (on every range type but Int32Range, Int64Range and DecimalRange), Shift, Expand, ToRangeSet()/ToInt32Set(), the set indexer and Compare on RangeLowerBoundComparer, RangeSet.LowerBoundComparer and any IComparer of ranges or sets each throw an EF translation exception inside a Where (an OrderBy for the comparer) and evaluate on the client with the in-memory answer inside a Select.",
"claim": "Length (on every range type but Int32Range and Int64Range), Values(), Values(step), Split, Clamp (on every range type but Int32Range, Int64Range and DecimalRange), Shift, Expand, ToRangeSet()/ToInt32Set(), the set indexer and Compare on RangeLowerBoundComparer, RangeSet.LowerBoundComparer and any IComparer of ranges or sets (decided by the operands, so also a comparer declared over IRange<T>, the non-generic IComparer and a comparer of value sets) each throw an EF translation exception inside a Where (an OrderBy for the comparer) and evaluate on the client with the in-memory answer inside a Select.",
"oracle": "exception",
"surfaces": [
"*Range.Length",
Expand Down
Loading