Skip to content

fix(store): support filter operators and apply limit after filtering - #242

Open
Harshil26202 wants to merge 1 commit into
redis-developer:mainfrom
Harshil26202:fix/store-search-filter-operators-and-limit
Open

Harshil26202 wants to merge 1 commit into
redis-developer:mainfrom
Harshil26202:fix/store-search-filter-operators-and-limit

Conversation

@Harshil26202

Copy link
Copy Markdown

Summary

RedisStore.search and AsyncRedisStore.asearch apply value filters client-side, after the Redis query. That has two effects that don't match LangGraph's BaseStore contract or InMemoryStore:

  1. Operator filters never match. BaseStore.search documents $eq, $ne, $gt, $gte, $lt and $lte. The Redis store compared the stored value with the operator dict itself (actual != expected), so for example filter={"score": {"$gt": 3}} silently returned an empty list. _get_filter_condition in base.py has the operators, but nothing calls it.
  2. limit and offset were applied before the filter. The Redis query only constrains the namespace and is paged with offset/limit, and the filter then runs on that page. So search(ns, filter={"tag": "a"}, limit=5) returns only the matches that happen to be in the first 5 documents of the namespace, often none.

Reproduction against redis:8, 40 items in one namespace, 15 of them with tag == "a":

store.search(("docs",), filter={"tag": "a"}, limit=5)        # 1 item, InMemoryStore returns 5
store.search(("docs",), filter={"n": {"$gt": 1}}, limit=100)  # [], InMemoryStore returns 22

Changes

  • base.py: a shared _matches_filter helper with the InMemoryStore semantics for the six operators and for nested dicts (matched key by key). A range operator on a missing or non-numeric value returns False instead of raising. The four copies of the inline filter loop in the sync and async vector and regular search paths now call it.
  • Regular (non-vector) search with a filter now pages through the namespace (FILTER_PAGE_SIZE = 500) and applies offset and limit to the filtered documents. Without a filter, the query is unchanged and Redis still does the paging.

Things I left unchanged on purpose

  • List values. This store treats filter={"tag": ["a", "b"]} as "any of". InMemoryStore (and the Postgres store) treat a list as an exact match on a list value. I kept the current behaviour so existing users aren't affected, but I'm happy to switch it if you'd rather match the reference.
  • Vector search with a filter still filters the limit nearest candidates, so it can also return fewer than limit results. Fixing that needs an over-fetch strategy, which felt like a design decision for you. Operator filters do work there now.

Testing

Integration tests against real Redis, in both tests/test_store.py and tests/test_async_store.py:

  • test_search_filter_with_comparison_operators
  • test_search_filter_applies_limit_and_offset_to_matching_items

All 4 fail on main and pass with the change. Every store and semantic search test file passes (85 passed). black and isort (the pinned versions) are clean on the changed files. mypy langgraph/store/redis reports the same single existing error as main (base.py:423).

AI disclosure: I found this with Claude Code by comparing RedisStore.search results with LangGraph's InMemoryStore on the same data and filters. It also drafted the fix and the tests.

RedisStore.search applied filters client-side with plain equality, so
operator filters such as {"score": {"$gt": 3}} compared the stored value
with the operator dict and never matched. The filter also ran after Redis
had already applied offset and limit, so search(filter=..., limit=5) could
return fewer matches than exist, often none.

Move the filter into a shared _matches_filter helper that follows
InMemoryStore's semantics for $eq, $ne, $gt, $gte, $lt, $lte and nested
dicts, and keeps the existing any-of behaviour for list values. When a
filter is set on a non-vector search, page through the namespace and
apply offset and limit to the filtered documents.

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant