T5: source-scoped export withholding in readRowsSince#374
Merged
philcunliffe merged 1 commit intoJul 22, 2026
Merged
Conversation
Extends readRowsSince (src/core/cache/storage.js) with a second optional resolver, sourceWithholdResolver, threaded the same way usagePolicyResolver already is: drop-but-advance, so a withheld row still moves the watermark (LLP 0070#incremental continuation semantics). createSourceWithholdResolver (src/core/cache/source-withhold.js) is a pure, catalog-agnostic factory over plain data (withheld source ids + a dataset -> attribution-column map). buildSourceWithholdResolver (src/core/runtime/source_withhold.js) does the boot-time reduction from classifyClientProvenance (T3) and the plugin catalog: the set of picker source ids classified 'local' on a machine with a central layer. bootKernel now builds the kernel runtime after the catalog and layered config are resolved (previously built at the top of the function) so it can pass this resolver through createKernelRuntime. Per-row matching uses the new attribution_column manifest field (T1). @hypaware/ai-gateway declares attribution_column: "client_name" on its ai_gateway_messages dataset contribution. A dataset with no declared attribution_column is never subject to source-scoped withholding, the conservative default matching local-only's existing design. Tests cover withhold-and-advance, the no-attribution-column passthrough default, no-resolver passthrough, projection-can't-bypass, and composition with the existing cwd-based local-only filter, all directly against readRowsSince. Task-Id: T5
philcunliffe
merged commit Jul 22, 2026
4a8aeca
into
integration/install-experience-overhaul
8 checks passed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Implements task T5 of install-experience-overhaul.
Extends
readRowsSincewith a second optional resolver,sourceWithholdResolver,threaded the same way
usagePolicyResolveralready is: drop-but-advance, so awithheld row still moves the watermark.
createSourceWithholdResolver(src/core/cache/source-withhold.js): pure,catalog-agnostic factory over plain data.
buildSourceWithholdResolver(src/core/runtime/source_withhold.js):boot-time reduction from
classifyClientProvenance(T3) + the plugincatalog, into the set of
'local'-classified picker source ids on amachine with a central layer.
bootKernelnow builds the kernel runtime after the catalog/layered configare resolved so it can wire the resolver through
createKernelRuntime.@hypaware/ai-gatewaydeclaresattribution_column: "client_name"on itsai_gateway_messagesdataset contribution (T1's manifest field). A datasetwith no declared
attribution_columnis never subject to source-scopedwithholding, the conservative default.
Tests directly against
readRowsSince: withhold-and-advance, theno-attribution-column passthrough default, no-resolver passthrough,
projection-can't-bypass, and composition with the existing cwd-based
local-only filter.
Test plan
npm test(7 pre-existing, unrelatedleave-command.test.jsfailuresconfirmed present on
origin/integration/install-experience-overhaulbefore this change; all new source-withholding tests pass)
npm run typecheckTask-Id: T5