Skip to content

graph: refuse unresolvable and non-directory parents - #494

Merged
typeless merged 4 commits into
mainfrom
fix/492-unresolved-parent
Sep 23, 2026
Merged

typeless merged 4 commits into
mainfrom
fix/492-unresolved-parent

Conversation

@typeless

@typeless typeless commented Sep 22, 2026 •

Copy link
Copy Markdown
Owner

Closes #492.

Four commits: #492 itself, plus the similar problems its review found.

1. add_file_node refuses an unresolvable parent id (#492)

A non-zero parent_dir that get_file_node cannot resolve is now InvalidNodeId, the code add_edge already returns. Before this, such a node was filed under the source root, and its name went into another node's dir_children bucket or past the end of dir_children. The new test observed that out-of-bounds write as a SIGSEGV. No production caller reaches this error.

2. glob: match files, never directories

glob_expand added any directory whose name matched, and the builder then bound it as a File node. Upstream's nl_add_path globs through tup_db_select_node_dir_glob with directories excluded. Red: gdir_* → { "gdir_sub", "gdir_top.txt" }. Now it returns only the file. New requirement: REQ-GLOB-FILES-ONLY.

3. graph: refuse a path under a node that is not a directory

Before this change:

  • : sub |> … followed by an output sub/x.o ran the command, which failed in the shell.
  • Two rules writing gen and gen/x both ran, and one failed.
  • A group sub/<g> under a file built with no diagnostic at all.

Upstream refuses all three in find_dir_tupid_dt_pg. add_file_node now refuses any parent that is not a Directory or GeneratedDir, with a message like Unable to create 'sub/x.o' because the file 'sub' is not a directory. New requirement: REQ-OUTPUT-UNDER-DIRECTORY.

The check exposed two producers that typed directories as files:

  • the glob (commit 2): putup's own test runner globs | ../e2e/fixtures/** and was refused;
  • resolve_input_node: a real directory named as a rule input became a File node, and a reviewer found six such Tupfiles that tup accepts and the check refused. It now asks the disk.

4. Delete resolve_phi_node

Nothing calls it, and its INVALID_NODE_ID fallback is also SOURCE_ROOT_ID.

Evidence

  • Each fix has a red-then-green test, and each red was observed before its fix.
  • make check → all tests passed (182569 assertions in 959 test cases, 32 e2e shards).
  • Bootstrap scripts regenerate unchanged.
  • Fable reviewed on two lenses, regression and correctness. Its 20 Tupfile fixtures exit identically under the installed pre-change binary and this branch.
  • The helloworld, busybox and bsp examples parse unchanged.

Not fixed here

🤖 Generated with Claude Code

https://claude.ai/code/session_01StgwMENEyfBnEoe4pvAdtQ

add_file_node took a non-zero parent_dir that get_file_node could not
resolve (an id never minted, a command-tagged id, a slot holding another
node) and filed the node's path under the source root, while still
inserting its name into dir_children[node_id::index(parent_dir)]. With
the tag stripped, that bucket belongs to some other node; with an index
past the new node's slot the write is out of bounds, since dir_children
is resized only to the new node's index and Vec::operator[] is
unchecked. The new test observed that as a SIGSEGV before the fix; the
issue had predicted it from the source without running it.

The parent is now resolved before anything is interned or minted, and an
unresolvable one is InvalidNodeId, the code add_edge already returns for
an unresolvable endpoint. get_file_node is the predicate rather than
validate_node_id, because the latter accepts a live command id.

No production caller reaches the error: every parent_dir comes from
ensure_file_node or find_by_dir_name on the same graph, or is 0, and the
index loader builds an Index, never a Graph. A parent that resolves but
is not a directory is still accepted; no caller passes one.

Verified red-then-green: before the fix the three sections SIGSEGV,
accept the command-id parent, and return DuplicateNode instead of
InvalidNodeId; after, make check passes (182542 assertions, 957 cases).

Ref: #492
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
@github-actions

github-actions Bot commented Sep 23, 2026 •

Copy link
Copy Markdown

PR metrics

Performance (gcc example, Linux)

Workload Instructions CPU time Page faults D1 miss LL miss Wall Peak RSS
parse 1760 M (+0.1%) 0.64 s 15.2 k 0.7% (+0.1pp) 0.1% (+0.1pp) 0.655 s 35.1 MB (-0.1MB)
dry-run 2486 M 0.75 s 16.8 k 0.8% (+0.2pp) 0% 0.772 s 41.3 MB (-0.1MB)

Deterministic signals: instructions (cachegrind-simulated instruction reads — exact across runs, no PMU needed), page faults, peak RSS, and the cachegrind D1/LL miss rates. CPU time is user+sys from time(1).

Internal statistics (gcc example, up-to-date dry run)

Metric Value
Tupfiles parsed 24
Commands 3545
Commands scheduled 0
Files checked 5834
Files changed 0
Files in index 6188
Graph edges 384927
Index size (bytes) 7869819
Implicit deps 344126
Hash computations 197
Hashes skipped (stat cache) 5636
Stat calls 5885
Parse time (ms) 579.1
Total time (ms) 756.2
Runner CPU AMD EPYC 9V74 80-Core Processor

Counters from putup -n --stat on the fully-built gcc example (up-to-date dry run): deterministic work measures — a jump in commands scheduled, hash computations, or stat calls is a real behavior change, not noise. Timings are the minimum over repeated runs, compared only against a baseline from the same CPU model; the counters are the regression signal.
Timing deltas suppressed: baseline ran on different hardware (INTEL(R) XEON(R) PLATINUM 8573C).

Binary size (Linux)

Binary .text .data .bss File
putup 603.2 KB (+0.1%) 2.4 KB 98.8 KB 716.3 KB (+0.6%)

Code churn (whole codebase, last 30d)

Files Lines written Still present Churned Churn rate
32 2522 2185 337 13.4%

Of the lines written across the codebase in the last 30 days, how many are already gone — work that was written and then discarded or rewritten inside the same window. This is the state of the tree including this PR, not a measure of the PR itself. Only code we write is counted: tests, examples, vendored and generated files, CI plumbing and prose are excluded. 1639 lines were deleted in the window in total, most of them older than it.

Where the churn is
File Lines written then discarded
src/parser/eval.cpp 77
src/graph/builder.cpp 72
src/graph/dag.cpp 56
src/index/entry.cpp 44
include/pup/core/token_list.hpp 23
include/pup/parser/eval.hpp 16
src/cli/cmd_build.cpp 8
src/index/reader.cpp 8
include/pup/core/instruction.hpp 7
src/core/instruction.cpp 7

Test coverage (lines)

Overall Median file Min file Max file
88.9% (+0.1pp) 96.5% (-0.2pp) 14.7% include/pup/parser/token.hpp 100.0% include/pup/core/arena.hpp

105 files · 17915/20154 lines covered

Deltas vs main@b595d6c70.

Updated for 1fd6b5a

typeless and others added 3 commits September 23, 2026 08:24
glob_expand added every directory entry whose name matched the pattern,
in both the flat and the recursive walk, and the builder then bound each
matched directory as a File node. A rule globbing `dir/**` took the
subdirectories of dir as inputs typed File, and anything later created
under one of them was parented to a file.

Upstream expands a rule's glob in nl_add_path through
tup_db_select_node_dir_glob with directories excluded, so only normal and
generated files match. glob_expand now skips an entry the directory read
reports as a directory; the recursive walk still descends into it.

Found when the directory check in the next commit refused putup's own
test runner, whose shards take `| ../e2e/fixtures/**`. A symlink to a
directory still matches, as before: the POSIX read types it Symlink.

Verified red-then-green: `gdir_*` returned { "gdir_sub", "gdir_top.txt" }
and `**` returned the subdirectory too; both now return the files only.
Bootstrap scripts regenerate unchanged.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
A Tupfile could put an output or a group under a source file, a
generated file or a group. add_file_node checked only that the parent id
resolved, and ensure_file_node returns whatever node a path already has
when asked for a directory. So `: sub |> ... ` followed by an output
`sub/x.o` ran the command and failed in the shell, and a group
`sub/<g>` under the file sub built with no diagnostic at all. Upstream
refuses all three in find_dir_tupid_dt_pg ("Unable to output to a
different directory because 'sub' is a normal file").

add_file_node is the one place a node acquires a parent, so it now
refuses any parent that is not a Directory or GeneratedDir, naming the
path and what occupies it. The code is InvalidNodeId, as for an
unresolvable parent: the id is not a valid parent.

The check trusts node types, and one producer typed them wrong:
resolve_input_node created a File node for any existing source or config
path, directories included. With the check alone, `: sub |> ls %f ...`
over a real directory followed by any path under sub was refused where
upstream accepts it. It now asks the disk whether the path is a
directory. Incremental behaviour of a directory input is unchanged:
against the installed pre-change binary, a build, a null build and an
edit of a file under the directory rebuild the same commands.

Not covered: putup learns a source path's type only once a rule names
it, so an output named under a file before any rule reads that file
creates a Directory node and still fails in the shell. Upstream knows
from its scan.

Verified red-then-green: before the check, the three e2e cases ran the
command, ran it and failed, and built; before the input fix, the
directory case was refused. make check passes (182569 assertions in 959
test cases), and a cross-model review's 20 Tupfile fixtures exit the same
under the old and the new binary.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
It returned INVALID_NODE_ID when the phi or its condition did not
resolve, and INVALID_NODE_ID is also SOURCE_ROOT_ID, so a failed lookup
read as the source root. No source, header or test calls it; it had
external linkage with no declaration, and DESIGN.md listed it. Deleting
it removes the ambiguous default instead of re-encoding it.

add_phi_node and get_phi_node have no production caller either, but
they are the phi model DESIGN.md describes, so they stay.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

add_file_node re-roots a node whose parent it cannot resolve instead of refusing it

1 participant