Repository navigation
✨ feat(index): link subdirectories to their index.md, not the bare directory - #11
Merged
Merged
Conversation
…ally Every bundle writer in internal/okf (node new, TouchModifiedFile, node mv and node promote link rewrites, migrate apply, Scaffold, index build, log append) used os.WriteFile, which truncates the target in place: a crash or a full disk mid-write left a half-written node behind. They now go through fsutil.WriteFileAtomic (temp file in the same directory, sync, rename). Each rewrite path is pinned by a hard-link test that fails on an in-place write; a v0.1 node (legacy timestamp, body # Citations) still migrates and touches as before.
…tomically okfconfig.Save, search.(*Store).Save and the migrate --plan writer used os.WriteFile, which truncates in place; a crash or a concurrent reader could observe a partial file. Route them through fsutil.WriteFileAtomic (temp file plus rename). Each swap is pinned by a hard-link test that fails against the in-place writer.
OKF §3 defines a bundle as its directory tree. node new checked only the lexical path, so a symlinked or junctioned directory inside the bundle could carry the write outside it; node edit did no check at all and launched the editor on ../outside.md. Both now resolve the user path through fsutil.ResolveInRoot before touching disk. A nested a/b/c path and a title carrying --- in prose still write as before.
ApplyMove joined the destination onto the root with no containment check, so node mv a ../x.md renamed the node out of the bundle (OKF §3) after already rewriting its inbound links. Both ends now go through fsutil.ResolveInRoot before any write; a control character in the destination is refused too, since the path is echoed into link text. A nested in-bundle destination still moves.
…red core package The tokenizer, stopword set and light stemmer move verbatim from internal/search to a stdlib-only internal/lexical package so core search can rank with the same chain without depending on the plugin. internal/search keeps its names as thin forwarders; --lexical-gate output and tests are unchanged.
okfctl search keeps its substring match set (PRD §6.3) and now orders it by BM25F (k1=1.2, b=0.75, per-token DF, title > tags > type > body), ties broken by path. --sort path reproduces the previous output byte for byte; --json gains a score field. Ranking is consumer-side (OKF §1); reserved files stay out (§3.1) and the open type vocabulary is ranked, never gated (§4.1). okf.RankBM25 and internal/lexical expose the ranking for hybrid fusion.
…one segment PromotePlan accepted any --name, so ../x, a/b or index wrote the promoted concept outside its directory, outside the bundle (OKF §3), or over a §3.1 reserved file; a line break in it landed raw in every rewritten link. --name must now be one ordinary, non-reserved file-name segment. PromoteApply confines every path a plan names (old index, new concept, rewrite targets) through fsutil.ResolveInRoot before any write. The default name, an ordinary name and one with --- in it are still accepted.
The plan file is user-editable between phases. MigrateApply joined each node path onto the root unchecked, so a plan naming ../victim.md rewrote a file outside the bundle (OKF §3), and wrote target_version verbatim into the line-keyed .okf sidecar, so a line break forged a second key (§12). Every plan node path now goes through fsutil.ResolveInRoot and a target carrying a control character is refused, both before any write.
AppendLog flattened a message to its first line by cutting at LF only. CommonMark also ends a line at a bare CR, so a message like "a<CR>## 1999-01-01" wrote a forged date heading into the §9 log. Cut at the first CR or LF.
…aths Four fuzz targets assert the write-safety properties on untrusted input: no injected frontmatter key from node new, no lost key or body from a modified refresh, a key-stable root index.md okf_version rewrite, and exactly one entry and heading per log append. Registered in the CI fuzz job.
Name ErrInvalidPath and the junction check in the ResolveInRoot contract, list the fuzz targets and tests as shipped, and correct the linkDir test comment: EvalSymlinks does not follow a junction.
…d honour the umask - On Windows an O_EXCL create under a directory junction fails with "file exists" (os.CreateTemp included), so every write into an in-bundle junctioned directory that ResolveInRoot accepts was refused. Open the temp without O_EXCL there, behind an Lstat check on a 64-bit random name. - Create the temp with perm instead of os.CreateTemp's forced 0600 plus a chmod, so a new file goes through the umask as it did with os.WriteFile. An existing file still keeps its own mode. - Retry a Windows rename briefly on access-denied, when another process holds the target open. - Sync the parent directory after the rename so the replace survives a crash (best effort; not possible on Windows). - Document that a hard-linked target is replaced rather than written through.
…efore any write - §3: ApplyMove now confines every node it rewrites, not only the two ends of the move, so a node reached through a directory link out of the bundle is never written through. - PromoteApply checks on disk that each destination is free before step 1 rewrites inbound links. The check used to run after the rewrites, so a refused promotion left links pointing at a concept that was never written. - §3.1: IsReservedPath cleans the path with the native separator first, so `node edit sub\index.md` on Windows is refused like sub/index.md.
…y platform - Go may fuse a multiply-add (arm64, ppc64, s390x), so raw float64 scores differed in their low bits by platform. --json was no longer CI-diffable, and near-equal hits could swap order. Round every score to 6 decimals where it is produced (lexical.Index.Scores), so both the output and the score-then-path order are the same everywhere. - Reword the search help and guide. BM25F weights a title hit over tags, type and body, but repeated body mentions still add up, so it does not guarantee a strict title > body order.
…rectory `index build` now writes a Subdirectories entry as `* [Wine](wine/index.md)` instead of `* [Wine](wine/)`. OKF §8 leaves the relative URL of an index entry open (its `subdir/` is an example, not a rule; §6.1 only asks for a standard Markdown path), and a file target resolves for readers that can only open files: an agent's read tool, a plain Markdown viewer. It also matches the reference producer (reference_agent/bundle/index.py). The on-disk parser accepts both spellings and keys them on the same directory, so a curator description written against a legacy `wine/` bullet carries over on the first rebuild. Existing bundles report stale once under `index check` until rebuilt. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
LGuichet
added a commit
that referenced
this pull request
Oct 30, 2026
… fix The index guide and the changelog describe the link form index build writes since the merge of #11, how an older index migrates, and the empty-null-key fix in this PR. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
okfctl index buildnow links each subdirectory entry to the child's ownindex.md(* [Wine](wine/index.md)) instead of the bare directory (* [Wine](wine/)). That way readers that can only open files (an agent's read tool, a plain Markdown viewer) can follow the link.Index output (OKF §8)
child/index.md. §8 leaves an entry's relative URL open (itssubdir/is an example, not a rule, and §6.1 only asks for a standard Markdown path), so both forms conform. The new one matches the reference producer,reference_agent/bundle/index.py, line 98.topics/index.mdand the legacytopics/, and treats both as the same subdirectory. A curator description written on an oldtopics/entry carries over to the new link on the first rebuild.Index rendering (
internal/okf/)reserved_lifecycle.go: renderschild/index.md, andsubdirBulletREaccepts both spellings ((?:index\.md)?).subdir_description_test.goaddsTestSubdirDescription_LegacyTrailingSlashLinkMigrates(positive control: a legacytopics/description survives astopics/index.md) andTestSubdirBulletRE_OnlyDirectoryLinks(negative control:nouns.md,topics/nouns.md,index.mdandtopics/index.md.bakare not taken for subdirectory entries). The existing fixtures move to the new link form.index_eol_test.go,index_shape_render_test.go,neighborhoods_test.go,nested_index_test.go,reserved_lifecycle_test.goCLI tests (
cmd/)index_nested_test.go,index_shape_test.go: pin the new link form throughokfctl index build.Evidence
index buildwrites* [Api](api/) - … (8 concepts · …).After: it writes
* [Api](api/index.md) - … (8 concepts · …).Spec-conformance suite (
go test ./internal/okf/ ./cmd/ -run Conformance):gofmt -l .is empty andgo vet ./...is clean. All other packages pass (okfctl-api,okfctl-search,agentplugin,apiserver,fsutil,lexical,search). Ininternal/okfandcmd, every index test passes. The 13 tests that fail locally fail identically onnext, because of this Windows host: plugins not found on PATH, shell-script editors,truemissing, and CRLF in git clones.-racewas not run locally (no C compiler on the host), so CI covers it.Real corpus: two production bundles, each copied and run with the
nextbinary (before) and this branch (after). "Subdir links" = Subdirectories entries in everyindex.mdafterindex build.dir/→dir/index.mdThe two
broken-linkfindings on nf525-lne predate this PR; that count not moving is the control.Merge Danger
Door: two-way
Blast Radius: index files
okfctl index checkonce (rc 0 → 1) untilindex buildis re-run; after that, rc 0. The rebuild rewrites every Subdirectories link and keeps curator descriptions.validateandlintare unaffected: lint's broken-link check scans concept nodes only, never index files, andnode promotealready resolved bothdir/anddir/index.md(promote.go).Not in this PR: no
--link-styleflag to keep emittingdir/, and the dated design records indocs/specs/still describe thechild/form they decided at the time.🤖 Generated with Claude Code