Skip to content

cognitive-shader-driver: dispatch stops materializing O(rows²); one SPOFC candidate per row - #1293

Merged
AdaWorldAPI merged 8 commits into
mainfrom
claude/llvm-codegen-polyfill-gni3cw
Sep 25, 2026
Merged

AdaWorldAPI merged 8 commits into
mainfrom
claude/llvm-codegen-polyfill-gni3cw

Conversation

@AdaWorldAPI

@AdaWorldAPI AdaWorldAPI commented Sep 25, 2026 •

Copy link
Copy Markdown
Owner

Two changes to one ShaderDriver::dispatch, found in order while tracing it end to end:

  1. It no longer materializes O(rows²) to return 8 hits. Output is unchanged.
  2. A row supported several ways is now one candidate carrying that support as evidence (a SPOFC record), instead of several candidate slots that cancel each other in the XOR braid. This one intentionally changes the output.

1. Materialization (output digest unchanged)

The module doc claims "no allocations beyond top-k + edges". A counting global allocator (tests/dispatch_trace.rs) measured one dispatch, after a warm-up:

rows before after part 1
16 23 KB, 44 allocs 1.2 KB, 5 allocs
256 5.3 MB, 530 allocs 2.2 KB, 5 allocs
  • Content pre-pass. It pushed two hits per resonant pair into a Vec (O(rows²)), then sorted it and cut it to 8. TopHits now keeps the best 8 as they arrive, with the same tie order as the stable sort.
  • p64-bridge cascade. It returned up to 256 candidates per row, of which the driver read 4. The new cascade_nearest::<K> returns only the K nearest and does not allocate. It shares one candidate walk with cascade, which is unchanged. CascadeHit is now Copy.
  • Evidence.
    • A digest over every crystal field (60 configurations × 3 dispatches) is identical before and after: 355d5fccbc424763.
    • Equivalence tests compare each replacement against the collect-sort-truncate it replaces.
    • Disable runs turn the allocation pin red.

2. SPOFC: one candidate per row, support kept

The problem.

  • Stage [4] builds cycle_fp = ⊕ R^{cycle_index} · row, and a row's cycle_index is its position.
  • Stage [3] offered every supporting relationship as its own slot: each content pair, and each P64 cascade hit.
  • So a row supported N ways contributed N identical terms, and XOR cancels them in pairs.
  • On empty predicate planes, 36 of 60 dispatches had a repeated row in top-k. In some of them, the strongest row vanished from cycle_fp.

The change. Each surviving row is now one candidate, and its relationships aggregate into a SPOFC record:

field value
subject the row
predicates union of every relationship's bits
object the best-resonance partner (Row(r) or a P64 Palette(target))
truth TruthU8 { f = best resonance, c = m·255/(m+k) }
support the relationship count m
  • Ranking uses the best resonance, and cycle_index stays the rotation basis. Each selected row enters the braid exactly once.
  • No SPOFC type existed in code, only in docs. The record follows the one code precedent, arm-discovery's CandidateTriple/TruthU8.
  • The confidence formula is shared, not copied: evidence_confidence_u8 was extracted from arm_to_truth_u8 and is now used by both.

Effects on what is emitted and cached (60 dispatches each, old vs new):

predicate planes cycle_fp changed edges changed top-k changed dispatches with a repeated row
non-empty 12 60 60 0 → 0
empty 48 48 48 36 → 0
  • With only the predicate union disabled, non-empty-plane edges change in 12/60 instead of 60/60. So 48 of those edge changes are the union adding CausalMask bits (predicates & 7) that the old per-slot emission dropped.
  • Anything keyed on cycle_fp or the emitted edges (caching, retrieval, replay) will see different values for these dispatches. That is the intended correction. The part-1 digest therefore no longer matches after part 2.
  • Allocation: one more, for the candidate table (6 in total). The count is still constant in rows, and bytes grow by 28 per surviving row (4-byte row id + 24-byte SPOFC record). dispatch_trace pins both.

Tests, each disable-verified red then green:

test what it covers disable that turns it red
repeated_support_is_one_candidate_and_survives_the_braid empty-plane regression: k = 8 distinct rows, cycle_fp equals the one-term-per-row XOR, the strongest row is present re-offering every relationship
repeated_support_is_retained_as_spofc_evidence support = n−1 per row, with the matching confidence not counting support
p64_targets_survive_as_spofc_support four P64 targets become one candidate with support 4, object a real Palette target not counting support
spofc_predicates_are_the_union_of_supporting_relationships predicate union dropping or overwriting the union

The union test is new because removing the union had left all 111 lib tests green. p64_target_identity_probe is re-pinned from "n > 1 identical edges" to exactly one edge.

Not in this PR (follow-up, draft)

  • The SPOFC record currently ends at a debug_assert. CE64 emission still packs s = row%256, p = 0, o = (row/4)%256 and f = c = resonance.
  • CE64's 24-bit S/P/O could hold the target as palette256³. The driver just doesn't write it there.
  • Writing SPOFC into CE64, and carrying that evidence across cycles without recounting it as fresh support, is the separate follow-up.
  • The prefilter row list (passed_rows, 4 bytes per row) is also still open.

Checks

  • cognitive-shader-driver: all 9 test binaries pass; clippy -D warnings and fmt are clean.
  • lance-graph-arm-discovery: 42 tests plus the new doctest pass.
    • Its --all-targets clippy fails on 14 doc-list lints at translator.rs:28-41, a comment this PR does not touch. The crate is not a workspace member.
  • p64-bridge: cascade tests pass.
    • One needless_range_loop clippy error in an existing test fails the same way without this PR.

Board entry: .claude/board/entries/2026-09-25-dispatch-materialization-trace.md.

🤖 Generated with Claude Code

https://claude.ai/code/session_019HnekoM1EidTwQLS3oFVFm


Generated by Claude Code

Summary by CodeRabbit

  • Performance
    • Cascade searches can retrieve the nearest candidates without generating and sorting every result.
    • Dispatch processing limits downstream work to the top eight ranked hits and keeps allocation growth bounded as rows increase.
  • Reliability
    • Multiple supporting relationships for a row are combined into one candidate, with support count and predicate evidence reflected in its ranking and confidence.
    • Equal-distance cascade results retain candidate order.
    • Duplicate target relationships from one source row now produce a single emitted result rather than repeated identical results.

A counting global allocator measures one dispatch (after warm-up) at
16..256 rows. Baseline: 23 KB / 44 allocs at 16 rows, 5.3 MB / 530
allocs at 256 rows, for an answer that is always <= 8 hits. Bytes grow
with the square of the population (the content pre-pass pushes two hits
per resonant pair into a Vec that is sorted and cut to 8); the
allocation count grows linearly (p64-bridge cascade returns a Vec per
row). The module doc claims no allocations beyond top-k + edges.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_019HnekoM1EidTwQLS3oFVFm
…scards

Two collections were built only to be cut down:

- The content pre-pass pushed two hits per resonant pair into a Vec
  (O(rows^2)) that was then stable-sorted and truncated to 8. TopHits
  keeps the best 8 as they arrive; a new hit goes after every kept hit
  whose resonance is not lower, so ties keep arrival order exactly as
  the stable sort did.
- p64-bridge's cascade returned every candidate (up to 256 per row) in a
  fresh Vec, sorted, of which the driver read 4. cascade_nearest::<K>
  keeps the K nearest as they arrive, allocation-free. cascade and
  cascade_nearest now share one private candidate walk, so the candidate
  logic exists once.

Measured by tests/dispatch_trace.rs (one dispatch after warm-up):

  rows   before              after
  16     23 KB, 44 allocs    1.2 KB, 5 allocs
  256    5.3 MB, 530 allocs  2.2 KB, 5 allocs

The remaining per-row growth is the prefilter's row list (4 bytes/row).
The test now pins a constant allocation count across 16..256 rows and at
most 4 bytes of growth per row.

Output is unchanged: a digest over every field of the crystal (60
configurations x 3 dispatches, including all 12 styles' paths through
Ordinal 0/5/11 and Auto, radii 50..MAX, 16..300 rows) is identical before
and after (355d5fccbc424763). TopHits and cascade_nearest each have an
equivalence test against the collect-sort-truncate they replace, on
tie-heavy inputs.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_019HnekoM1EidTwQLS3oFVFm
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_019HnekoM1EidTwQLS3oFVFm
…ndSpace)

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_019HnekoM1EidTwQLS3oFVFm
@coderabbitai

coderabbitai Bot commented Sep 25, 2026 •

Copy link
Copy Markdown

Review in Change Stack →

Navigate logical layers of code changes, visualize relationships, and explore their blast radius.

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info
⚙️ Run configuration

Configuration used: Organization UI

Review profile: CHILL

Plan: Essentials

Run ID: f8d9ab10-b6f9-4a6d-b493-a0310a969dff

📥 Commits

Reviewing files that changed from the base of the PR and between 8a85595 and ee4e145.

📒 Files selected for processing (6)
  • .claude/board/entries/2026-09-25-dispatch-materialization-trace.md
  • crates/cognitive-shader-driver/Cargo.toml
  • crates/cognitive-shader-driver/src/driver.rs
  • crates/cognitive-shader-driver/tests/dispatch_trace.rs
  • crates/cognitive-shader-driver/tests/p64_target_identity_probe.rs
  • crates/lance-graph-arm-discovery/src/translator.rs
🚧 Files skipped from review as they are similar to previous changes (1)
  • .claude/board/entries/2026-09-25-dispatch-materialization-trace.md

Included review availability: 0 reviews are currently available. Your included PR review attempts over the past 7 days set your current allowance at 2 reviews per hour.


📝 Walkthrough

Walkthrough

Dispatch now aggregates relationship support into one candidate per prefiltered row and retains up to eight ranked hits. Cascade search exposes a fixed-size nearest-hit API. Tests and board entries describe aggregation, allocation measurements, and open trace items.

Changes

Dispatch candidate materialization

Layer / File(s) Summary
Evidence confidence calculation
crates/lance-graph-arm-discovery/src/translator.rs, crates/cognitive-shader-driver/Cargo.toml
evidence_confidence_u8 calculates confidence with integer arithmetic. arm_to_truth_u8 uses the helper, and the driver adds the crate dependency.
Bounded cascade candidate selection
crates/p64-bridge/src/lib.rs
cascade_nearest<K> visits candidates through a shared visitor and returns up to K hits in distance order. Tests compare its results with prefixes from cascade, including ties.
Per-row candidate aggregation and dispatch ranking
crates/cognitive-shader-driver/src/driver.rs, crates/cognitive-shader-driver/tests/p64_target_identity_probe.rs
Content matches and cascade results add support to one candidate per prefiltered row. The driver ranks candidates by best relationship resonance and retains up to eight hits for downstream processing. Tests cover aggregation, evidence, predicate unions, and emitted CE64 counts.
Allocation checks and trace updates
crates/cognitive-shader-driver/tests/dispatch_trace.rs, .claude/board/entries/2026-09-25-dispatch-materialization-trace.md, .claude/board/entries/README.md, .claude/board/SUPERSESSION-INDEX.md
The allocation test updates its per-row allowance. Board entries record aggregation details, allocation observations, open trace items, and index count changes.

Priority: ➖ Normal

Estimated code review effort: 3 (Moderate) | ~25 minutes

Change: Bug fix

Sequence Diagram(s)

sequenceDiagram
  participant ShaderDriver
  participant CognitiveShader
  participant CandidateTable
  participant TopHits
  ShaderDriver->>CognitiveShader: Request up to four nearest cascade hits
  CognitiveShader-->>ShaderDriver: Return cascade hits
  ShaderDriver->>CandidateTable: Add content and cascade relationship support
  CandidateTable-->>ShaderDriver: Return supported row candidates
  ShaderDriver->>TopHits: Rank candidates and retain up to eight hits
  TopHits-->>ShaderDriver: Return retained hits
Loading

Suggested reviewers: claude

Merge Risk: ⚪ Minimal · up to ee4e1

The dispatch change is mergeable after normal checks; no actionable behavior regression remains identified.

🚥 Pre-merge checks | ✅ 4 | ❌ 1

❌ Failed checks (1 warning)

Check name Status Explanation Resolution
Docstring Coverage ⚠️ Warning Docstring coverage is 62.50% which is insufficient. The required threshold is 80.00%. Docstring coverage is scoped to functions touched by this diff. Analyzed 40 functions across 5 files. (2 skipped: … Write docstrings for the functions missing them to satisfy the coverage threshold.
✅ Passed checks (4 passed)
Check name Status Explanation
Linked Issues check ✅ Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check ✅ Passed Check skipped because no linked issues were found for this pull request.
Description Check ✅ Passed Check skipped - CodeRabbit’s high-level summary is enabled.
Title check ✅ Passed The title clearly summarizes the main change: dispatch avoids O(rows²) materialization and creates one SPOFC candidate per row.
Full details: Docstring Coverage

Explanation

Docstring coverage is 62.50% which is insufficient. The required threshold is 80.00%. Docstring coverage is scoped to functions touched by this diff. Analyzed 40 functions across 5 files. (2 skipped: 2 unsupported.)

  • Fix all pre-merge checks with AI
✨ Finishing Touches 💡 1
📝 Generate docstrings 💡
  • Commit to this branch
  • Create a new PR

A rabbit counts each row with care,
One candidate gathers support to spare.
Four nearby hits hop into view,
Eight ranked results pass onward through.
The tests keep tie order neat,
While board notes track each measured beat.

Comment @coderabbitai help to get the list of available commands.

@cursor

cursor Bot commented Sep 25, 2026

Copy link
Copy Markdown

Bugbot couldn't run - usage limit reached

Bugbot is counted against Cursor usage for this user or team, and this run hit a usage or spend limit.

A user or team admin can review and increase usage limits in the Cursor dashboard.

(requestId: serverGenReqId_b5729680-bfb9-41c0-9a0d-6a1855c17220)

@AdaWorldAPI
AdaWorldAPI marked this pull request as ready for review September 25, 2026 15:01
…olyfill-gni3cw

# Conflicts:
#	.claude/board/entries/README.md
… evidence

Stage [3] offered every supporting relationship (content pre-pass pair,
each P64 cascade hit) as its own candidate slot. A row supported N ways
took N top-k slots with one cycle_index, and the XOR braid cancelled
repeated (row, basis) pairs — on empty predicate planes the strongest row
could vanish from cycle_fp entirely (36/60 dispatches carried a repeated
row in top_k).

Now each surviving row is one candidate. Its supporting relationships
aggregate into a SPOFC record: predicate union, best-resonance partner
(Row or P64 Palette target) as object, support count as evidence, and
TruthU8 { f = best resonance, c = m*255/(m+k) } via the arm-discovery
convention (evidence_confidence_u8, extracted from arm_to_truth_u8).
Ranking uses best resonance; cycle_index stays the rotation basis.

Measured over 60 dispatches (old vs new):
  non-empty planes: cycle_fp 12/60, edges 60/60, top_k 60/60 changed
  empty planes:     cycle_fp 48/60, edges 48/60, top_k 48/60 changed
Disabling only the predicate union drops the non-empty edge change to
12/60, so the other 48 are the CausalMask bits of the union.

CE64 emission is unchanged in shape: the target stays in the SPOFC
record; writing it into CE64's 24-bit S/P/O is the follow-up.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_019HnekoM1EidTwQLS3oFVFm
The union had no falsifier: removing it left all 111 lib tests green,
because the first relationship's bits were kept instead.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_019HnekoM1EidTwQLS3oFVFm
@cursor

cursor Bot commented Sep 25, 2026

Copy link
Copy Markdown

Bugbot couldn't run - usage limit reached

Bugbot is counted against Cursor usage for this user or team, and this run hit a usage or spend limit.

A user or team admin can review and increase usage limits in the Cursor dashboard.

(requestId: serverGenReqId_efd91bf4-40f9-4cf5-abb5-b17e981c4cf1)

@AdaWorldAPI AdaWorldAPI changed the title cognitive-shader-driver: a dispatch no longer materializes O(rows²) to return 8 hits cognitive-shader-driver: dispatch stops materializing O(rows²); one SPOFC candidate per row Sep 25, 2026
@AdaWorldAPI
AdaWorldAPI merged commit a1c9cae into main Sep 25, 2026
11 checks passed
AdaWorldAPI added a commit that referenced this pull request Sep 25, 2026
…l-gni3cw-ce64-spofc

cognitive-shader-driver: emit CE64 from the SPOFC record (follow-up to #1293)
AdaWorldAPI added a commit that referenced this pull request Sep 26, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants