Repository navigation
fix(store): support filter operators and apply limit after filtering - #242
Open
Harshil26202 wants to merge 1 commit into
Open
Harshil26202 wants to merge 1 commit into
Harshil26202 wants to merge 1 commit into
Conversation
RedisStore.search applied filters client-side with plain equality, so
operator filters such as {"score": {"$gt": 3}} compared the stored value
with the operator dict and never matched. The filter also ran after Redis
had already applied offset and limit, so search(filter=..., limit=5) could
return fewer matches than exist, often none.
Move the filter into a shared _matches_filter helper that follows
InMemoryStore's semantics for $eq, $ne, $gt, $gte, $lt, $lte and nested
dicts, and keeps the existing any-of behaviour for list values. When a
filter is set on a non-vector search, page through the namespace and
apply offset and limit to the filtered documents.
This branch has not been deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
RedisStore.searchandAsyncRedisStore.asearchapply value filters client-side, after the Redis query. That has two effects that don't match LangGraph'sBaseStorecontract orInMemoryStore:BaseStore.searchdocuments$eq,$ne,$gt,$gte,$ltand$lte. The Redis store compared the stored value with the operator dict itself (actual != expected), so for examplefilter={"score": {"$gt": 3}}silently returned an empty list._get_filter_conditioninbase.pyhas the operators, but nothing calls it.limitandoffsetwere applied before the filter. The Redis query only constrains the namespace and is paged withoffset/limit, and the filter then runs on that page. Sosearch(ns, filter={"tag": "a"}, limit=5)returns only the matches that happen to be in the first 5 documents of the namespace, often none.Reproduction against
redis:8, 40 items in one namespace, 15 of them withtag == "a":Changes
base.py: a shared_matches_filterhelper with theInMemoryStoresemantics for the six operators and for nested dicts (matched key by key). A range operator on a missing or non-numeric value returnsFalseinstead of raising. The four copies of the inline filter loop in the sync and async vector and regular search paths now call it.FILTER_PAGE_SIZE = 500) and appliesoffsetandlimitto the filtered documents. Without a filter, the query is unchanged and Redis still does the paging.Things I left unchanged on purpose
filter={"tag": ["a", "b"]}as "any of".InMemoryStore(and the Postgres store) treat a list as an exact match on a list value. I kept the current behaviour so existing users aren't affected, but I'm happy to switch it if you'd rather match the reference.limitnearest candidates, so it can also return fewer thanlimitresults. Fixing that needs an over-fetch strategy, which felt like a design decision for you. Operator filters do work there now.Testing
Integration tests against real Redis, in both
tests/test_store.pyandtests/test_async_store.py:test_search_filter_with_comparison_operatorstest_search_filter_applies_limit_and_offset_to_matching_itemsAll 4 fail on
mainand pass with the change. Every store and semantic search test file passes (85 passed).blackandisort(the pinned versions) are clean on the changed files.mypy langgraph/store/redisreports the same single existing error asmain(base.py:423).AI disclosure: I found this with Claude Code by comparing
RedisStore.searchresults with LangGraph'sInMemoryStoreon the same data and filters. It also drafted the fix and the tests.