Repository navigation
graph: refuse unresolvable and non-directory parents - #494
Conversation
add_file_node took a non-zero parent_dir that get_file_node could not resolve (an id never minted, a command-tagged id, a slot holding another node) and filed the node's path under the source root, while still inserting its name into dir_children[node_id::index(parent_dir)]. With the tag stripped, that bucket belongs to some other node; with an index past the new node's slot the write is out of bounds, since dir_children is resized only to the new node's index and Vec::operator[] is unchecked. The new test observed that as a SIGSEGV before the fix; the issue had predicted it from the source without running it. The parent is now resolved before anything is interned or minted, and an unresolvable one is InvalidNodeId, the code add_edge already returns for an unresolvable endpoint. get_file_node is the predicate rather than validate_node_id, because the latter accepts a live command id. No production caller reaches the error: every parent_dir comes from ensure_file_node or find_by_dir_name on the same graph, or is 0, and the index loader builds an Index, never a Graph. A parent that resolves but is not a directory is still accepted; no caller passes one. Verified red-then-green: before the fix the three sections SIGSEGV, accept the command-id parent, and return DuplicateNode instead of InvalidNodeId; after, make check passes (182542 assertions, 957 cases). Ref: #492 Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
PR metricsPerformance (gcc example, Linux)
Deterministic signals: instructions (cachegrind-simulated instruction reads — exact across runs, no PMU needed), page faults, peak RSS, and the cachegrind D1/LL miss rates. CPU time is user+sys from time(1). Internal statistics (gcc example, up-to-date dry run)
Counters from Binary size (Linux)
Code churn (whole codebase, last 30d)
Of the lines written across the codebase in the last 30 days, how many are already gone — work that was written and then discarded or rewritten inside the same window. This is the state of the tree including this PR, not a measure of the PR itself. Only code we write is counted: tests, examples, vendored and generated files, CI plumbing and prose are excluded. 1639 lines were deleted in the window in total, most of them older than it. Where the churn is
Test coverage (lines)
105 files · 17915/20154 lines covered Deltas vs main@b595d6c70. Updated for 1fd6b5a |
glob_expand added every directory entry whose name matched the pattern,
in both the flat and the recursive walk, and the builder then bound each
matched directory as a File node. A rule globbing `dir/**` took the
subdirectories of dir as inputs typed File, and anything later created
under one of them was parented to a file.
Upstream expands a rule's glob in nl_add_path through
tup_db_select_node_dir_glob with directories excluded, so only normal and
generated files match. glob_expand now skips an entry the directory read
reports as a directory; the recursive walk still descends into it.
Found when the directory check in the next commit refused putup's own
test runner, whose shards take `| ../e2e/fixtures/**`. A symlink to a
directory still matches, as before: the POSIX read types it Symlink.
Verified red-then-green: `gdir_*` returned { "gdir_sub", "gdir_top.txt" }
and `**` returned the subdirectory too; both now return the files only.
Bootstrap scripts regenerate unchanged.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
A Tupfile could put an output or a group under a source file, a
generated file or a group. add_file_node checked only that the parent id
resolved, and ensure_file_node returns whatever node a path already has
when asked for a directory. So `: sub |> ... ` followed by an output
`sub/x.o` ran the command and failed in the shell, and a group
`sub/<g>` under the file sub built with no diagnostic at all. Upstream
refuses all three in find_dir_tupid_dt_pg ("Unable to output to a
different directory because 'sub' is a normal file").
add_file_node is the one place a node acquires a parent, so it now
refuses any parent that is not a Directory or GeneratedDir, naming the
path and what occupies it. The code is InvalidNodeId, as for an
unresolvable parent: the id is not a valid parent.
The check trusts node types, and one producer typed them wrong:
resolve_input_node created a File node for any existing source or config
path, directories included. With the check alone, `: sub |> ls %f ...`
over a real directory followed by any path under sub was refused where
upstream accepts it. It now asks the disk whether the path is a
directory. Incremental behaviour of a directory input is unchanged:
against the installed pre-change binary, a build, a null build and an
edit of a file under the directory rebuild the same commands.
Not covered: putup learns a source path's type only once a rule names
it, so an output named under a file before any rule reads that file
creates a Directory node and still fails in the shell. Upstream knows
from its scan.
Verified red-then-green: before the check, the three e2e cases ran the
command, ran it and failed, and built; before the input fix, the
directory case was refused. make check passes (182569 assertions in 959
test cases), and a cross-model review's 20 Tupfile fixtures exit the same
under the old and the new binary.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
It returned INVALID_NODE_ID when the phi or its condition did not resolve, and INVALID_NODE_ID is also SOURCE_ROOT_ID, so a failed lookup read as the source root. No source, header or test calls it; it had external linkage with no declaration, and DESIGN.md listed it. Deleting it removes the ambiguous default instead of re-encoding it. add_phi_node and get_phi_node have no production caller either, but they are the phi model DESIGN.md describes, so they stay. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Closes #492.
Four commits: #492 itself, plus the similar problems its review found.
1.
add_file_noderefuses an unresolvable parent id (#492)A non-zero
parent_dirthatget_file_nodecannot resolve is nowInvalidNodeId, the codeadd_edgealready returns. Before this, such a node was filed under the source root, and its name went into another node'sdir_childrenbucket or past the end ofdir_children. The new test observed that out-of-bounds write as a SIGSEGV. No production caller reaches this error.2.
glob: match files, never directoriesglob_expandadded any directory whose name matched, and the builder then bound it as aFilenode. Upstream'snl_add_pathglobs throughtup_db_select_node_dir_globwith directories excluded. Red:gdir_*→{ "gdir_sub", "gdir_top.txt" }. Now it returns only the file. New requirement:REQ-GLOB-FILES-ONLY.3.
graph: refuse a path under a node that is not a directoryBefore this change:
: sub |> …followed by an outputsub/x.oran the command, which failed in the shell.genandgen/xboth ran, and one failed.sub/<g>under a file built with no diagnostic at all.Upstream refuses all three in
find_dir_tupid_dt_pg.add_file_nodenow refuses any parent that is not aDirectoryorGeneratedDir, with a message likeUnable to create 'sub/x.o' because the file 'sub' is not a directory. New requirement:REQ-OUTPUT-UNDER-DIRECTORY.The check exposed two producers that typed directories as files:
| ../e2e/fixtures/**and was refused;resolve_input_node: a real directory named as a rule input became aFilenode, and a reviewer found six such Tupfiles that tup accepts and the check refused. It now asks the disk.4. Delete
resolve_phi_nodeNothing calls it, and its
INVALID_NODE_IDfallback is alsoSOURCE_ROOT_ID.Evidence
make check→all tests passed (182569 assertions in 959 test cases, 32 e2e shards).Not fixed here
Directorynode and still fails in the shell. Relatedly,ensure_file_node(Generated)returns an existingDirectoryunchanged, where tup reports "Attempting to insert … as a generated node".get_parent_dirreturns0for an unknown id. That is the family's shared contract, not one function's defect:get<NodeType>returnsFile,get<NodeFlags>returnsNone, and so on. The root cause is thatINVALID_NODE_ID == SOURCE_ROOT_ID. All production callers pass live ids.is_guard_satisfiedleft as is. All 14 callers pass live command ids, and "not a command, so no guards" is a correct answer.🤖 Generated with Claude Code
https://claude.ai/code/session_01StgwMENEyfBnEoe4pvAdtQ