Repository navigation
feat: secondary indexes on scalar properties (indexed=True) - #12
Merged
Merged
Conversation
Declaring indexed=True on String/Integer/Float/Boolean maintains a
persistent per-value ID list (key format _fi:{Type}:{field}:{value},
same pattern as relation reverse indexes), kept up to date by the
property descriptor on create/update/delete. New Entity classmethods:
- find_by(field, value, from_id, count): equality lookup with cursor
pagination, cost proportional to matches instead of max_id
- count_by(field, value): match count without loading entities
- rebuild_field_index(field, from_id, batch): resumable backfill for
entities that predate the index, bounded per call so IC canisters can
spread it across update calls without hitting instruction limits
Entity hydration from storage now skips hooks, validation, and index
maintenance in Property.__set__ (indexes are already intact), mirroring
the existing Relation fast path.
The indexed flag is intentionally excluded from the schema descriptor:
toggling it is an operational change, not a data-shape change.
Closes #11
Co-authored-by: Cursor <cursoragent@cursor.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
String(..., indexed=True)(and Integer/Float/Boolean) maintains a persistent per-value ID list under_fi:{Type}:{field}:{value}keys — the same storage pattern as the existing relation reverse indexes.Entityclassmethods:find_by(field, value, from_id=1, count=50)— equality lookup with cursor pagination, cost proportional to matches instead ofmax_idcount_by(field, value)— match count without loading entitiesrebuild_field_index(field, from_id=1, batch=200)— resumable backfill for entities that predate the index, bounded per call for IC instruction-limit safetyEntity.delete()cleans up index entries; hydration from storage skips hooks/validation/index writes inProperty.__set__(mirrors the Relation fast path).indexedflag is intentionally excluded from the schema descriptor, so toggling it never triggers upgrade-compatibility errors.Closes #11
Test plan
tests/src/tests/test_indexed_fields.py(15 cases): lookups, non-string values, update/None/delete maintenance, persistence across load, pagination cursor semantics, resumable + idempotent rebuild, unindexed-field errorsrun_test.sh,entrypoint.sh(IC), and the test canistermain.pyrun_linters.shclean (black, isort, flake8, mypy)Made with Cursor