Repository navigation
Fix UseOffsetPaging on a nested EF collection - #573
Merged
Merged
Conversation
A paged collection on a parent object is part of the parent's projection, so EF has to translate the paging. Items were paged with our Skip/Take(int?) helpers, which EF does not know, failing with "The LINQ expression 'p_X => new Dynamic_items...' could not be translated". Use System.Linq's Skip/Take for a collection that is not an IQueryable. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
…Page Without a take, items was paged with Take(int.MaxValue). EF pages a nested collection with ROW_NUMBER() and `row <= skip + take`, which overflows an int on SQL Server and Postgres for any skip > 0. Take int.MaxValue - skip instead. hasNextPage on a nested collection called EnumerableExtensions.PageHasNext, which EF cannot translate, so it loaded every column of the whole collection for every parent and evaluated it in memory. Build the same check from System.Linq calls (take.HasValue && Skip(skip + take).Any()) so it is an EXISTS query. PagingSqlTests pins both against the SQL EF executes. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Problem
UseOffsetPaging()on a field that is not on the root query type fails when its collection comes from EF, for example a navigation property:{ actor(id: 1) { movies(take: 10) { totalItems items { name } } } }This happens with any nested offset paging over an EF collection, whether the parent is a single object or a list and whether or not the collection is ordered. It fails on every version I tried, from 5.7.1 through
main.Cause
OffsetPagingItemsExtensionpagesitemswith EntityGraphQL's ownEnumerableExtensions.Skip/Take(int?). On a root field that is harmless, because the call doesn't depend on a row and EF evaluates it before translating. A nested collection depends on the parent row and is part of the parent's projection, so EF has to translate the call, and it doesn't know these helpers.Fix
A collection that is not an
IQueryableis now paged withSystem.Linq'sSkip/Take. A nullskipbecomes0. A nulltakebecomesint.MaxValue - skip, which is "the rest of the collection" as theint?helpers did. It isn't plainint.MaxValuebecause EF pages a nested collection withROW_NUMBER()androw <= skip + take, and that sum would overflow aninton SQL Server and Postgres for anyskip > 0. TheIQueryablepath is unchanged.hasNextPageon a nested collection had the same root cause. It calledEnumerableExtensions.PageHasNext, which EF can't translate, so EF loaded every column of the whole collection for every parent and evaluated it in memory. It's now built fromSystem.Linqcalls (take.HasValue && source.Skip(skip + take).Any()), which EF translates to anEXISTSquery.Tests
PagingTests(EF/SQLite) gets two tests: a nested page withtotalItems/hasNextPage/itemsunder a single-object field, and one withhasNextPage/itemsonly (theEXISTSpath, withouttotalItems) under a list. Both fail without the fix.PagingSqlTestsgets two tests that check the SQL EF runs: nestedhasNextPageis anEXISTSand doesn't select unrequested columns, andskipwithouttakehas no2147483647in it. SQLite's integers are 64-bit, so it can't show the overflow itself. Both fail without the second commit. The EntityGraphQL, AspNet and EF test projects all pass on net8.0, net9.0 and net10.0.Not covered
Nested
UseConnectionPaging()over an EF collection fails the same way. Its edges use the sameSkip/Take(int?)helpers, but cursors are also assigned in memory byConnectionHelper.ApplyCursors, so it still fails once Skip/Take is fixed. That needs a bigger change, so I've left it for a separate fix.🤖 Generated with Claude Code