Repository navigation
Stream large quantized tables with CPU/RSS evidence and raw fallback - #41
Merged
Merged
Conversation
initial-d
changed the title
initial-d
marked this pull request as ready for review
October 31, 2026 11:40
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Exporting the 1,855,907 × 256 trained table as INT8 previously peaked at 9,118 MiB RSS by materializing whole FP32/encoded/reconstructed arrays. The file exporter now streams codec/hash/audit chunks: the identical native artifact peaks at 61.7 MiB (147.9× lower). Readers can gather into reusable FP32 and encoded buffers; native descriptors and optional planner indexes still grow with table size.
The CPU study checks all original-ID encoded/reconstructed bytes across eight full layouts, two actual trained-row prefix builds and 54 randomized fresh-process query workers. FP16 native saves 52.29% complete backend bytes versus raw FP32, with a broad-query cost. Native INT8 expands 1.23% versus raw INT8; the optional identity-row
native="auto"policy compares table + codec metadata and falls back to raw. Full-table auto export and reader verification pass, with hashes matching the verified raw control. Tiny native blocks improve miss latency but remain slower/larger than raw INT8. FP16/INT8 are lossy; this study measures vector distortion, without downstream model-quality or GPU claims.Raw results, commands, build/source/table hashes, per-process latency ranges, resource comparisons, failure details and plots are in the evidence report. The existing DATrie/mark-compact contributions remain separate from this identity-row codec study. Fixed filesystem warming and shared-host limitations stay close to results.
Validation: 141 local tests (26 dependency/platform skips); Linux's earlier full suite passes 140 tests (17 skips), and all 12 final codec/capacity/auto tests pass. No GPU processes, training or external-data downloads. The source-pinned SSH launcher timeout and direct-buffer accounting correction are preserved explicitly.