Repository navigation
Conversation
…d the §8 index suffix Lift the per-directory aggregation that `index build` renders as the subdirectory shape suffix (OKF §8 progressive disclosure) out of dirShape into a structured DirShape: own and subtree concept counts, the sorted distinct `type` values (§7.4 open vocabulary, kept as written), and the §4.1 shared tags with a truncation flag. Shape computes it for one directory and Shapes for every IndexDirs entry. dirShape now renders FROM the struct, so the index suffix and any other consumer (the PRD §6.1 "neighborhoods" view of `bundle info`) cannot drift apart. No behavior change to the rendered suffix. Tests cover own-vs-subtree counting, own-concept-only types and tags, tag-cap truncation, byte-equality between Shape and the rendered suffix, a build→validate round trip, and Shapes covering every index dir.
…ex.md OKF §12 defines where a bundle declares the version it targets: an `okf_version` key in the bundle-root index.md frontmatter block. Load read the version only from the .okf sidecar, which is okfctl's own pin and not part of the spec, so a spec-conforming bundle carrying the marker and no sidecar was reported as SpecVersion whatever it declared. index build already resolved the marker first (bundleRootOkfVersion); load and build disagreed about the same bundle. Resolve Bundle.OkfVersion in the order index build writes the marker: the root-index marker, else the sidecar, else SpecVersion. Only the root index counts (§12 permits the marker nowhere else). The declared string is reported as declared: per §12 an unrecognised version is consumed best-effort, never rejected. Tests pin both directions: marker-only, marker-over-sidecar, bare scalar and unrecognised values are read (positive); a nested index.md marker, a blank marker and a sidecar-only bundle leave the result unchanged (negative). The conformance suite gains the consumer half of the §12 round trip: the marker index build preserves is the one Load reads back.
…red core package The tokenizer, stopword set and light stemmer move verbatim from internal/search to a stdlib-only internal/lexical package so core search can rank with the same chain without depending on the plugin. internal/search keeps its names as thin forwarders; --lexical-gate output and tests are unchanged.
okfctl search keeps its substring match set (PRD §6.3) and now orders it by BM25F (k1=1.2, b=0.75, per-token DF, title > tags > type > body), ties broken by path. --sort path reproduces the previous output byte for byte; --json gains a score field. Ranking is consumer-side (OKF §1); reserved files stay out (§3.1) and the open type vocabulary is ranked, never gated (§4.1). okf.RankBM25 and internal/lexical expose the ranking for hybrid fusion.
…ally Every bundle writer in internal/okf (node new, TouchModifiedFile, node mv and node promote link rewrites, migrate apply, Scaffold, index build, log append) used os.WriteFile, which truncates the target in place: a crash or a full disk mid-write left a half-written node behind. They now go through fsutil.WriteFileAtomic (temp file in the same directory, sync, rename). Each rewrite path is pinned by a hard-link test that fails on an in-place write; a v0.1 node (legacy timestamp, body # Citations) still migrates and touches as before.
…tomically okfconfig.Save, search.(*Store).Save and the migrate --plan writer used os.WriteFile, which truncates in place; a crash or a concurrent reader could observe a partial file. Route them through fsutil.WriteFileAtomic (temp file plus rename). Each swap is pinned by a hard-link test that fails against the in-place writer.
OKF §3 defines a bundle as its directory tree. node new checked only the lexical path, so a symlinked or junctioned directory inside the bundle could carry the write outside it; node edit did no check at all and launched the editor on ../outside.md. Both now resolve the user path through fsutil.ResolveInRoot before touching disk. A nested a/b/c path and a title carrying --- in prose still write as before.
ApplyMove joined the destination onto the root with no containment check, so node mv a ../x.md renamed the node out of the bundle (OKF §3) after already rewriting its inbound links. Both ends now go through fsutil.ResolveInRoot before any write; a control character in the destination is refused too, since the path is echoed into link text. A nested in-bundle destination still moves.
…one segment PromotePlan accepted any --name, so ../x, a/b or index wrote the promoted concept outside its directory, outside the bundle (OKF §3), or over a §3.1 reserved file; a line break in it landed raw in every rewritten link. --name must now be one ordinary, non-reserved file-name segment. PromoteApply confines every path a plan names (old index, new concept, rewrite targets) through fsutil.ResolveInRoot before any write. The default name, an ordinary name and one with --- in it are still accepted.
The plan file is user-editable between phases. MigrateApply joined each node path onto the root unchecked, so a plan naming ../victim.md rewrote a file outside the bundle (OKF §3), and wrote target_version verbatim into the line-keyed .okf sidecar, so a line break forged a second key (§12). Every plan node path now goes through fsutil.ResolveInRoot and a target carrying a control character is refused, both before any write.
AppendLog flattened a message to its first line by cutting at LF only. CommonMark also ends a line at a bare CR, so a message like "a<CR>## 1999-01-01" wrote a forged date heading into the §9 log. Cut at the first CR or LF.
…aths Four fuzz targets assert the write-safety properties on untrusted input: no injected frontmatter key from node new, no lost key or body from a modified refresh, a key-stable root index.md okf_version rewrite, and exactly one entry and heading per log append. Registered in the CI fuzz job.
Name ErrInvalidPath and the junction check in the ResolveInRoot contract, list the fuzz targets and tests as shipped, and correct the linkDir test comment: EvalSymlinks does not follow a junction.
…d honour the umask - On Windows an O_EXCL create under a directory junction fails with "file exists" (os.CreateTemp included), so every write into an in-bundle junctioned directory that ResolveInRoot accepts was refused. Open the temp without O_EXCL there, behind an Lstat check on a 64-bit random name. - Create the temp with perm instead of os.CreateTemp's forced 0600 plus a chmod, so a new file goes through the umask as it did with os.WriteFile. An existing file still keeps its own mode. - Retry a Windows rename briefly on access-denied, when another process holds the target open. - Sync the parent directory after the rename so the replace survives a crash (best effort; not possible on Windows). - Document that a hard-linked target is replaced rather than written through.
…efore any write - §3: ApplyMove now confines every node it rewrites, not only the two ends of the move, so a node reached through a directory link out of the bundle is never written through. - PromoteApply checks on disk that each destination is free before step 1 rewrites inbound links. The check used to run after the rewrites, so a refused promotion left links pointing at a concept that was never written. - §3.1: IsReservedPath cleans the path with the native separator first, so `node edit sub\index.md` on Windows is refused like sub/index.md.
With core.autocrlf=true (the Windows default) every text file checks out as CRLF, so gofmt -l listed all 218 Go files, the command-reference drift test failed on docs/commands/README.md, and index check saw every index as stale. Pin eol=lf for all text files so a clone is LF regardless of local config. No stored content changes: git add --renormalize is a no-op.
…y platform - Go may fuse a multiply-add (arm64, ppc64, s390x), so raw float64 scores differed in their low bits by platform. --json was no longer CI-diffable, and near-equal hits could swap order. Round every score to 6 decimals where it is produced (lexical.Index.Scores), so both the output and the score-then-path order are the same everywhere. - Reword the search help and guide. BM25F weights a title hit over tags, type and body, but repeated body mentions still add up, so it does not guarantee a strict title > body order.
…rectory `index build` now writes a Subdirectories entry as `* [Wine](wine/index.md)` instead of `* [Wine](wine/)`. OKF §8 leaves the relative URL of an index entry open (its `subdir/` is an example, not a rule; §6.1 only asks for a standard Markdown path), and a file target resolves for readers that can only open files: an agent's read tool, a plain Markdown viewer. It also matches the reference producer (reference_agent/bundle/index.py). The on-disk parser accepts both spellings and keys them on the same directory, so a curator description written against a legacy `wine/` bullet carries over on the first rebuild. Existing bundles report stale once under `index check` until rebuilt. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
- confine.go: wrap the underlying cause with %w beside ErrEscapesRoot, so errors.Is still matches ErrEscapesRoot and now also sees the cause. - confine.go: trim leading separators with TrimLeftFunc; on Unix the cutset `string(filepath.Separator)+"/"` repeated '/' (SA1024). Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
An explicit empty key (`? `) is a null key, which a map decode drops. The `modified` refresh round-trips the block through yaml.v3, whose encoder writes that empty plain scalar back as a quoted empty string, so the touched node gained a "" key it never had. FuzzTouchModifiedRoundTrip caught it in CI (`touch added keys: [] -> [ modified]`). Rewrite such keys as `~` before encoding, at every depth. A key that already is a quoted empty string is real data and keeps its quotes (negative control). The failing input joins the fuzz seeds. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
… fix The index guide and the changelog describe the link form index build writes since the merge of #11, how an older index migrates, and the empty-null-key fix in this PR. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Decoding into a yaml.Node skips map-level checks such as a duplicate key, so the modified refresh rewrote an invalid block (`ype: 0` then `ype:`) instead of refusing it and leaving the file byte-identical. FuzzTouchModifiedRoundTrip caught it in CI on #12. Run the frontmatter parser first and return its error; the input joins the fuzz seeds. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
index build rewrote every index.md on every run, and an index with mixed line endings (an editor or agent wrote LF lines into a CRLF file) came out LF even when nothing changed. On a core.autocrlf=true checkout git then reports the file modified with an empty diff, since its size no longer matches the CRLF size git recorded. Compare the render with the on-disk file the way index check does (line endings are not content) and skip the write when they match. A changed index still keeps CRLF when it was pure CRLF, and falls back to LF otherwise. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…racter bumpRootIndexMarker left the scalar style to yaml.v3, which writes a target such as a tab then a newline as a literal block whose tab-led line no parser reads back, so migrate --apply could leave an unparseable bundle-root index.md. FuzzRootIndexMarker caught it in CI on #13. Force double-quoted style when the target holds a control character; every other target is written as before. The input joins the fuzz seeds. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Owner
Author
|
CI check done: all 10 checks pass on the rebased next (53524f6). Closing without merging; main stays a mirror of cwest/okfctl main. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
CI check for the rebased
next(do not merge).nextis now cwest/okfctlmain(5bb1fac) plus the fork's 29 commits replayed as linear history. CI runs only on pushes tomainand on pull requests, so this draft gets-race, golangci-lint and fuzz to run on it.Dropped in the replay (already upstream)
a3f25e4and its KStem follow-up9827da4are covered by ✨ feat(lint): add tag-hygiene beside type-hygiene on a shared fold cwest/okfctl#167, which includes the KStem plural guard.2ca0898is replaced by upstream's 🐛 fix(index): handle CRLF index.md — normalize on read, preserve on write cwest/okfctl#205.Conflicts resolved (
internal/okf/)reserved_lifecycle.go:AppendLogkeeps upstream's CRLF handling (🐛 fix(log): handle CRLF log.md — normalize on read, preserve on write cwest/okfctl#207) and writes through the fork's atomic writer.WriteIndexkeeps upstream's CRLF preserve-on-write and adds the fork's skip of an index whose content is current.index_eol_test.go: upstream's version, with its subdirectory links moved to thedir/index.mdform, plus the fork'sTestWriteIndex_UnchangedIndexIsNotRewrittenin upstream's helper style.tagfold.gohunk, since that file does not exist upstream.Evidence
nextatce38a7e(fork history merged with an older upstream).After:
nextat53524f6.git diff ce38a7e 53524f6contains only upstream changes since1fee902plus the resolutions above.Local checks on Windows:
gofmt -l .is empty,go vet ./...is clean, and the-run Conformancesuite passes. The full suite fails only on tests that fail identically on the oldnexton this host (plugin and editor exec, CRLF git clones). The twoTestConnect_*tests now pass, thanks to cwest#204.Real corpus: copies of three production bundles, old
nextbuild against this branch.The cluster order changes because upstream's cwest#167 sorts clusters on the separator-insensitive key, so
e-reportingnow followsen16931.Merge Danger
Door: two-way
Blast Radius: none (draft)
mainis a mirror of upstreammain.🤖 Generated with Claude Code