Repository navigation
Conversation
|
Important Review skippedAuto reviews are limited based on label configuration. 🏷️ Required labels (at least one): 1
Please check the settings in the CodeRabbit UI or the
 |
peterkc
force-pushed
the
ci/fix/study-cross-repo-bridges
branch
3 times, most recently
from
October 27, 2026 00:21
3d57203 to
56c3635
Compare
The go.mod module-path bridge linked Go calls between tracked repositories, but file-to-package import edges into a tracked module stayed on dep::<module> terminals, even after a full warmup. The resolver's own comments say tracked sibling imports resolve through the file graph; the dep fallback is for modules with no matching files. The fallback also collapses subpackage paths to the module ID. Add an import-only bridge. It reads the retained import statement nodes for the exact package path, picks the provider package with the same module and workspace rules as the call bridge, and points the import edge at the provider's package file with the origin and confidence of other resolved cross-repo imports. Dependency contract nodes and their consumes edges are unchanged. It runs on full and mutation-frontier passes and reuses the module revision retry state, and it handles consumers whose go.mod is in a subdirectory. Re-indexing a provider file deletes incoming import edges (existing behavior in SQLite file eviction). The bridge records which consumer import statements point into each provider and re-creates those edges when the provider is re-indexed. After a restart the record is rebuilt once, lazily, by filtering import node IDs by path before loading any file; later edits do not scan. Package identities are cached per module revision. Repository sweeps filter by import path before hydrating files and release the resolve lock between batches. Tests cover both track orders, IndexAll, consumer edits and re-index, subdirectory go.mod consumers, provider file and repository re-index, an unchanged warm restart followed by a provider edit, duplicate paths and counts, and lock handling under panic. They fail before this change on memory and SQLite stores.
The import recovery recorded a repository's imports of its own module, so each save in a repository that imports its own packages reloaded nearly every file, even with one tracked repository. Record and restore only imports whose provider is a different tracked repository, and skip restore when fewer than two repositories are in the graph. The provider package cache was not cleared on the whole-graph fallback paths, so a consumer edit after a provider file was deleted could bind the import to the deleted file. Clear the cache on those paths and apply import rebinds through the same live-target check as call rebinds. An import lifted from an external:: fallback is now counted as resolved, not unresolved. The one-time lazy seed runs from the public entry points before the resolve lock is taken, so no lookup helper releases its caller's lock. New tests fail before this change.
Recovery of retained Go imports now takes the caller's context and stops between batches when it is cancelled. The presence check matched only the statement line, so two imports on one line were treated as one. It now matches the import path. It also accepts an import bound to a longer tracked module (for example example.com/lib/v2 next to example.com/lib), which avoids a duplicate edge on every full pass. File nodes load only when a same-line edge is a dep:: terminal.
Use indexed module paths and retain import identity through provider eviction. Recheck file bindings on module changes and bound incoming import recovery. Verify provider lifecycles against independent cold indexes.
…budget Recognize caller-local replace targets before restoring import rows. Keep import recovery and restart seeding on a separate bounded budget so incoming call binding and terminal retries still run.
…very Seed retained import identities once without hydrating files or rewriting edges. Keep provider import recovery on its own bounded budget so a large workspace cannot disable recovery after a warm restart.
Recover retained imports for the existing module-change repository set, not only the saved repository. Bound this recovery independently so an import refusal cannot suppress incoming call binding or terminal retry.
A Rust `use other_crate::x` import of a crate whose source is another
tracked repository stayed external. Directory matching does not know
crate roots, Cargo package names or item paths, and Gortex had no map
from crate names to repositories.
Map each tracked repository's Cargo crates (root package and listed,
non-excluded workspace members) to their exposed name ([lib].name, else
the package name with dashes as underscores) and root file ([lib].path,
else src/lib.rs). An external Rust import links only when the crate that
owns the importing file declares the dependency in its own Cargo.toml
([dependencies], dev, build and target tables, and workspace-inherited
entries), including `alias = { package = ... }` renames, and exactly one
eligible tracked crate provides it. The rest of the path resolves to a
module file in that crate, falling back to the crate root for items.
Workspace policy is checked against the provider crate, not the alias.
The retry runs in the full cross-repo pass and in the mutation-frontier
pass, so a link survives an edit to the importing file. It is skipped
when no tracked repository has a Cargo.toml. Generic directory matching
and other languages are unchanged.
Limit: a link stays after the consumer removes the dependency until the
importing file is re-indexed.
Tests index two tracked repositories on memory and SQLite stores and
cover declared, renamed, member, lib-renamed and edited-consumer cases,
plus refusals for undeclared, ambiguous, excluded-member and
cross-workspace imports. The positive cases fail before this change.
peterkc
force-pushed
the
ci/fix/study-cross-repo-bridges
branch
from
October 30, 2026 16:16
56c3635 to
8f31199
Compare
zzet#806's whole-shadow retirement and save restub paths can delete imports owned by unchanged consumers. Keep the exact pre-link stub on Cargo imports, park those imports and their cross-repo calls before provider eviction, and carry consumer files through the bounded cross-repo tail. Retry declared consumers after IndexRepo, track and full_retrack. Keep warmup's cross-repo-only files separate from provider derived plans. Mirror the approved npm lifecycle shape without reparsing importers. Cover both stores, both track orders, saves, full replacement, warm full_retrack, untrack/retrack, unchanged repositories and linked calls.
Record the pre-link function or method target instead of deriving a stub from the resolved symbol ID. Park only incoming calls to evicted targets, using the existing view rather than re-reading SQLite edge copies. Keep zzet#806's save carry/restub decisions and dataflow sites. Apply the recorded call stub only when that existing path chooses to restub, and pass forced file eviction's existing batch through the deletion core. Compare imports and calls with a fresh cold index on both stores and track orders, including associated functions, methods and forced eviction.
Use main's installed manifest-tree boundary for Cargo roots, workspace members and nearest-manifest checks instead of reading another revision's checkout. Keep metadata-only targets out of save-time import parking. Retain IndexRepo's cross-repo consumer result and skip its runtime-track resolver tail when the batch defers resolution. Cover both failures and snapshot manifests, and make warm-retrack tests use the configured order.
Preserve Rust file additions on IndexResult and registered warm receipts. Consume them in the existing scoped or full resolve after registration, including provider-first warmup and multiple changed providers. Restrict scoped recovery to marked imports into those providers. Keep the native matcher and write only changed targets. Record additions installed by the incremental fallback path too.
A Python `import pkg.mod` or `from pkg import mod` of a package whose source is another tracked repository stayed external. Calls through such imports already linked, but the import edges did not: directory matching cannot map a dotted module to src/pkg/mod.py or pkg/__init__.py. Map each tracked repository's Python distribution ([project].name or [tool.poetry].name, PEP 503-normalized) to its top-level import packages under the repository root or src/. An import links only when the importing repository declares that distribution in its own pyproject.toml ([project] dependencies and optional dependencies, Poetry dependencies and groups) or requirements*.txt, and exactly one eligible tracked repository provides the top-level package. The module file is chosen with the existing Python module matching (.py and __init__.py). The import stays local or external when the importing repository has its own files under that top-level name (including namespace folders without __init__.py), for standard-library names, and for root tooling files such as setup.py and conftest.py. Per-repository attribution keeps the dotted import path on the edge so the retry can use it. The retry runs in the full cross-repo pass, with light edge scans and paged hydration, and in the mutation-frontier pass, so a link survives an edit to the importing file. It does nothing with fewer than two tracked repositories. Limits: .pyi-only modules do not link, because stubs are not indexed. A link stays after the dependency is removed from the manifest until the importing file is re-indexed. Dotted modules do not match cross-workspace module rules, so such imports stay external. Tests index two tracked repositories on memory and SQLite stores. The positive cases fail before this change.
The full-pass retry kept every tracked-import edge before processing any page. With SQLite's projection scanner it now processes bounded pages; other stores drain the light scan first, so no live cursor stays open while the retry reads and writes the graph. The consumer's local names now include both the literal top-level name and the src-layout name. A root `src` package and a package under `src/` each stay local, even when `src/__init__.py` exists.
Read manifests, requirements and package layout through the owning indexer's manifest tree. Carry added-file importer candidates in CrossRepoResolveFiles so deferred and provider-first warmup can resolve after all repositories register. Keep the existing matcher and avoid consumer-frontier scans on ordinary saves. A file-only addition cannot trigger a whole-graph fallback.
Record successful new Python files on the fallback path, matching the structural batch path. This preserves consumer retry when a refresh read changes after the probe. Cover the fallback with a deterministic transform on memory and SQLite, then compare import and call bindings with a cold index.
Build import bridge evidence before resolving calls from the same provider save.
Remove obsolete package provenance in every call-parking route and finish the direct Python rebind handoff. Verify complete rebound edge payloads on both stores.
Exclude root and src packages named test or tests at provider discovery and at Python call resolution. Keep consumer-local bindings and normal modules. pytest test paths and other non-exported packages are not covered yet.
Give import-authorized functions and path-qualified constructors the same evidence as wildcard methods. Keep the exact-module rejection unchanged.
Reuse source file payloads and batch target ownership for cross-repo import counts. Assert native work counts and unchanged-pass behavior on both stores.
Compare complete Cargo and Python edge payloads, and cover rename and back and untrack-only. Repair the provenance, terminal-stamp, cold-order and purge-retry gaps this exposed. Fallback and synthesis run before the tracked-provider bridge, as in a cold index, so bridge refusals still win. After provider removal, consumers match a consumer-only cold index.
Name the seven bridge resolution metadata keys and exercise them in a small mixed-language two-repository fixture.
A full pass restores every import without a budget, so it no longer counts deferrals and can mark recovery complete. Keep the bounded deferral on mutation passes only. Gate the Cargo and Python terminal reconcile on the incoming refusal alone, not on a Go import deferral.
A row that keeps its unresolved target needs a live source edge, not a node for the target. Real retargets still need a live concrete target. Both stores now keep the same restub provenance.
Receipt and watcher callers pass the files they already resolved, and the bridge step does only the remaining outgoing work. A Python save now runs one incoming resolve instead of two, on both stores.
Unrelated non-Go saves no longer seed imports or retry module dependencies. Go source saves and Go build-list inputs (go.mod, go.sum, go.work, go.work.sum, vendor/modules.txt) still seed once per revision and retry.
Snapshots check size and mtime with the two-second racy rule and take the stamp before the first stat. File inventory, roots, config, manifests and glob directories invalidate them, and an unreadable directory marks the snapshot unknown. Stable passes read no manifests.
When the manifest snapshots show no tracked Python or Cargo dependency, added files no longer scan every import edge. An unknown snapshot still scans. Gaining a dependency reopens recovery and matches a cold index; workspaces with a declared bridge still scan per added file.
Read light inbound identities and load full edges only for foreign import and call sites carrying a bridge marker.
Read the caller's imports from the bridge index and memoize each package's imports for the pass. A later export edit on the same resolver still sees the new target.
Count only persisted, unregistered graph repositories in an active batch. Configured repositories that never register no longer keep the extra import scans running.
The re-export memo was written by resolveEdge workers during a full pass, which races and can crash the daemon. Fill it while building the reachable repo index, so workers only read it, like the other per-pass caches.
Install cargoCrateLookup, which keeps the cheap root count, so the Cargo gates skip Rust work again in workspaces with fewer than two Cargo roots and cold output is unchanged there.
Pass the files the master resolver saw, not the union with affected files, so affected Python and Rust referrers still get the pre-bridge outgoing pass and match a cold index.
When a mutation pass defers Go import recovery without an incoming refusal, request the existing full cross-repo fallback, so later Go saves do not repeat the all-repo recheck until a restart.
A root package has no slash in its directory, so every root crate with a dependency counted as a bridge. Path dependencies on its own workspace members no longer do.
Run the Go module retry whenever re-queued batches are pending, without reseeding, so a dropped reindex or failed untrack retry does not wait for the next Go save or full pass.
Fix comments left stale by the snapshot refactor and state when a snapshot is reused: stat stamps, the racy window, the file-set revision and the config revision.
…e pass A provider save above the recovery budget logs and skips, as before the full-pass continuation, and no longer marks the pass refused or reopens the revision. Only the revision-wide consumer recheck can request the one full continuation. Consumers of a provider with more than 16,384 registered imports keep stale import targets until a full pass.
Replace the env-gated indexer test with an external resolver integration test that shrinks the recovery budget through export_test.go and drives the public indexer entry point. It fails on both stores when the full continuation is removed.
Collect the distinct package IDs and read their edges with one GetOutEdgesByNodeIDs call per full pass instead of one point read per __init__.py.
Iterate provider repos in sorted order so a shared budget always skips the same provider, and name the skipped repo in the warning: its consumers stay stale until the next full pass.
Share the recheck kind list between production and the counting stores, so a changed kind list fails the zero-recheck assertions instead of passing without measuring anything.
peterkc
force-pushed
the
ci/fix/study-cross-repo-bridges
branch
from
October 30, 2026 16:45
8f31199 to
4998f59
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Fork-only draft to run CI for pi-gortex's fix/study-cross-repo-bridges at b7aca1a (Go cross-repo imports fix), plus one fork-only commit with the small zzet#941 test layout. Do not merge.