Skip to content

graph: keep nodes that name no path out of the path namespace - #489

Merged
typeless merged 1 commit into
mainfrom
fix/486-path-addressable
Sep 21, 2026
Merged

typeless merged 1 commit into
mainfrom
fix/486-path-addressable

Conversation

@typeless

Copy link
Copy Markdown
Owner

Closes #486.

A tup.config key that matched a source file's name made that file unreachable. Four shapes, each reproduced against an A/B control differing only in the config key's name:

project at HEAD
CONFIG_LICENSE=y, in-source, rule reads LICENSE Nothing to do (up to date) for ever, out.txt frozen
same, out-of-tree -B build FAILED: cp build/LICENSE build/out.txt, cannot stat
CONFIG_dep.h=y, header discovered by gcc -MD header edits invisible, any build shape
CONFIG_LICENSE=y, LICENSE reached by : foreach *E same freeze

The issue was filed naming only the first. The out-of-tree shape is the common one and fails outright, which is the practical severity here.

Cause

add_file_node interns <parent>/<name> into a PathId and registers it in path_to_node and the parent's dir_children for every node it is handed — and a config variable is such a node, named with the bare key and parented to the config directory. So the variable owns a path. A rule input resolving through ensure_file_node, or a discovered dependency joining through the index's files_by_path, finds it and binds to it, and nothing ever stats the file behind it. The out-of-tree shape differs only in which path the variable lands on: the config directory is the build directory, and a variant build grounds an ungrounded input there.

Fix

Path-addressability becomes a property of the node's kind rather than of whichever producer created it. is_path_addressable answers false for Variable, Condition and Phi — nodes standing for a value rather than for something on disk — and add_file_node and files_by_path are the two sites that consult it. The switch is exhaustive, so a new node type is a build error here rather than a silent entry in the path namespace.

The gate is on the lookup maps only. A node still interns its PathId, because get_full_path reads it and show graph labels every non-command node with that. Withholding it printed every config var, env var and @TUP_TOOL_* fingerprint as an unlabelled node — a regression the pair review caught, since no test asserts those labels.

The config-variable reuse branch in add_tupfile is deleted with it: after the gate it can find no Variable, and the only node it could still adopt is a same-named file or directory, which would drop the config value out of command identity.

Upstream

tup does not have this bug, and reaches disjointness a different way: it parents a TUP_NODE_VAR under the tup.config node rather than under its directory (tup_db_get_tup_config_tent → tup_db_read_vars → add_var), so its unique(dir, name) key separates them. That does not transfer to putup, whose path namespace is lexical — a Tupfile can spell tup.config/LICENSE, so a parent alone would relocate the bug rather than close it. Gating on kind does not depend on what a name can be spelled. This was the design an adversarial review refuted, on a measured repro.

Costs

INDEX_VERSION 27 → 28, and it is load-bearing rather than bookkeeping: without it a project whose index predates this change stays wedged for ever, because Index::compute_paths reconstructs a Variable entry's path from parent and name whatever the writer stored, and reconcile_input_set then finds that stale entry at the source's path and never marks the real file as new. Verified both ways — wedged on a v27 index without the bump, repaired with it. One forced rebuild.

files_by_path's contract narrowed from "every entry with a non-empty path" to "every path-addressable entry", so the property test pinning the old contract is updated to the new one and strengthened to assert an unaddressable kind is not reachable by path.

Verification

All four scenarios observed RED — with the gate disabled each fails with its own signature (Nothing to do (up to date), cannot stat 'build/LICENSE', Nothing to do., Nothing to do (up to date)) — and green with it restored.

make check: rc=0, 182495 assertions in 955 test cases across 32 e2e shards.

Residuals

  • Group nodes stay path-addressable, because group resolution goes through find_by_dir_name. A file whose basename is literally <g> can still become the group node; that reproduces identically before and after this change, and wants its own issue.
  • The property test's generator names every entry n<i>, so no Variable ever shares a path with a File in it — the new assertion witnesses re-admission, not shadowing. The four e2e scenarios carry that half.
  • add_file_node discards the duplicate-path signal, so two nodes at one path silently alias #487 is still open underneath all of this: add_file_node discards SortedPairVec::insert's duplicate-key return, so a collision that does slip through is still silent.

🤖 Generated with Claude Code

https://claude.ai/code/session_01StgwMENEyfBnEoe4pvAdtQ

A `tup.config` key that matched a source file's name made that file
unreachable. Three shapes, all reproduced with an A/B control differing only
in the key's name:

  CONFIG_LICENSE=y, in-source, rule reads LICENSE
      -> "Nothing to do (up to date)" for ever, out.txt frozen at its first
         contents
  CONFIG_LICENSE=y, out-of-tree -B build, same rule
      -> "FAILED: cp build/LICENSE build/out.txt", cannot stat
  CONFIG_dep.h=y, header discovered by gcc -MD
      -> header edits invisible, on any build shape
  CONFIG_LICENSE=y, LICENSE reached by a glob rather than named
      -> same freeze

Root cause: `add_file_node` interns `<parent>/<name>` into a PathId and
registers it in `path_to_node` and the parent's `dir_children` for every node
it is handed, and a config variable is such a node - named with the bare key,
parented to the config directory. So the variable owns a path. A rule input
resolving through `ensure_file_node`, or a discovered dependency joining
through the index's `files_by_path`, finds it and binds to it, and nothing
ever stats the file behind it. The out-of-tree shape differs only in which
path the variable lands on: the config directory is the build directory, and
a variant build grounds an ungrounded input there.

Path-addressability is now a property of the node's kind rather than of
whichever producer created it. `is_path_addressable` answers false for
`Variable`, `Condition` and `Phi` - nodes that stand for a value rather than
for something on disk - and `add_file_node` and `files_by_path` are the two
sites that consult it. The switch is exhaustive, so a new node type is a
build error here rather than a silent entry in the path namespace.

The gate is on the lookup maps only. A node still interns its PathId, because
`get_full_path` reads it and `show graph` labels every non-command node with
that; withholding it printed each config var, env var and tool fingerprint as
an unlabelled node. Being unfindable by path is the invariant; having no path
was an accident of where the gate first went.

The config-variable reuse branch in `add_tupfile` is deleted with it. After
the gate it can find no Variable, and the only node it could still adopt is a
same-named file or directory - which would drop the config value out of
command identity, since `compute_command_signature` folds a sticky source
only when its type is `Variable`.

Upstream reaches the same disjointness a different way: tup parents a
`TUP_NODE_VAR` under the `tup.config` node itself rather than under its
directory (`tup_db_get_tup_config_tent`, db.c:440-459, through
`tup_db_read_vars`, updater.c:358 and :427, to `add_var`, db.c:4979), so its
`unique(dir, name)` key separates them. That does not transfer to putup,
whose path namespace is lexical: a Tupfile can spell `tup.config/LICENSE`,
so a parent alone would relocate the bug rather than close it. Gating on kind
does not depend on what a name can be spelled.

INDEX_VERSION goes 27 to 28. It is load-bearing, not bookkeeping: without it
a project whose index was written before this change stays wedged for ever,
because `Index::compute_paths` reconstructs a Variable entry's path from
parent and name whatever the writer stored, and `reconcile_input_set` then
finds that stale entry at the source's path and never marks the real file as
new. Verified both ways - wedged on a v27 index without the bump, repaired
with it.

`files_by_path`'s contract narrowed from "every entry with a non-empty path"
to "every path-addressable entry", so the property test that pinned the old
contract is updated to the new one and strengthened to assert that an
unaddressable kind is not reachable by path.

The version ledger in `format.hpp` gets no entry: it records layout changes
that leave an older record unparseable, which is why 23, 24, 25 and 27 have
none either. This bump is a wrong-join, so its reason is here instead.

Residual: the property test's generator names every entry `n<i>`, so no
Variable ever shares a path with a File in it and the new assertion only
witnesses re-admission, not shadowing; the four scenarios carry that half.
`Group` nodes stay path-addressable, because group resolution goes
through `find_by_dir_name`; a path spelled to collide with one is not
reachable through today's lexer. #487 is still open underneath all of this -
`add_file_node` discards `SortedPairVec::insert`'s duplicate-key return, so
a collision that does slip through is still silent.

Verified red-then-green on all three shapes, each failing on the parent
commit for its own reason. make check: 182488 assertions in 954 test cases
across 32 e2e shards.

Ref: #486

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
@github-actions

Copy link
Copy Markdown

PR metrics

Performance (gcc example, Linux)

Workload Instructions CPU time Page faults D1 miss LL miss Wall Peak RSS
parse 1758 M 0.65 s 15.2 k 0.7% (+0.1pp) 0.1% (+0.1pp) 0.636 s 35.2 MB (+0.1MB)
dry-run 2484 M 0.74 s 16.8 k 0.8% (+0.2pp) 0% 0.743 s 41.3 MB (-0.1MB)

Deterministic signals: instructions (cachegrind-simulated instruction reads — exact across runs, no PMU needed), page faults, peak RSS, and the cachegrind D1/LL miss rates. CPU time is user+sys from time(1).

Internal statistics (gcc example, up-to-date dry run)

Metric Value
Tupfiles parsed 24
Commands 3545
Commands scheduled 0
Files checked 5834
Files changed 0
Files in index 6188
Graph edges 384927
Index size (bytes) 7869819
Implicit deps 344126
Hash computations 197
Hashes skipped (stat cache) 5636
Stat calls 5885
Parse time (ms) 555.7
Total time (ms) 718.8
Runner CPU AMD EPYC 7763 64-Core Processor

Counters from putup -n --stat on the fully-built gcc example (up-to-date dry run): deterministic work measures — a jump in commands scheduled, hash computations, or stat calls is a real behavior change, not noise. Timings are the minimum over repeated runs, compared only against a baseline from the same CPU model; the counters are the regression signal.
Timing deltas suppressed: baseline ran on different hardware (Intel(R) Xeon(R) 6973P-C).

Binary size (Linux)

Binary .text .data .bss File
putup 601.2 KB 2.4 KB 98.8 KB 711.8 KB

Code churn (whole codebase, last 30d)

Files Lines written Still present Churned Churn rate
32 2439 2105 334 13.7%

Of the lines written across the codebase in the last 30 days, how many are already gone — work that was written and then discarded or rewritten inside the same window. This is the state of the tree including this PR, not a measure of the PR itself. Only code we write is counted: tests, examples, vendored and generated files, CI plumbing and prose are excluded. 1606 lines were deleted in the window in total, most of them older than it.

Where the churn is
File Lines written then discarded
src/parser/eval.cpp 77
src/graph/builder.cpp 72
src/graph/dag.cpp 53
src/index/entry.cpp 44
include/pup/core/token_list.hpp 23
include/pup/parser/eval.hpp 16
src/cli/cmd_build.cpp 8
src/index/reader.cpp 8
include/pup/core/instruction.hpp 7
src/core/instruction.cpp 7

Test coverage (lines)

Overall Median file Min file Max file
88.9% 96.9% 14.7% include/pup/parser/token.hpp 100.0% include/pup/core/arena.hpp

105 files · 17880/20109 lines covered

Deltas vs main@04ed389f0.

Updated for ddd1f94

@typeless
typeless merged commit dddfb56 into main Sep 21, 2026
13 checks passed
@typeless
typeless deleted the fix/486-path-addressable branch September 21, 2026 11:27
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

A config variable aliases a same-named source file in the config directory, and edits to that file become invisible

1 participant