Repository navigation
operands: spell the rule's order-only inputs for %i and %Ni, refuse them elsewhere - #475
Merged
Merged
Conversation
…hem elsewhere putup treated %i as a second spelling of %f at all three expansion sites, so `: a.c | z.h |> echo i=[%i] > %o |> out.txt` wrote i=[a.c] where tup writes i=[z.h]. Upstream's tup_printf walks the rule's order_only_inputs for %i, refuses `%i is only valid in a command string` when no such list was passed (outputs, extra outputs, the input list) and `%i used in rule pattern and no order-only input files were specified` when the list is empty; %Ni selects the N-th written token from the same list and never refuses on emptiness. putup also refused %Ni outright as unsupported (#426's placeholder). The rule's order-only inputs now become a third operand list beside the inputs and outputs: a TokenList in PatternFlags for the parse-time render, on CommandNode for the command that runs and is hashed, and in the index operand record for the recorded command. The %i atom kind is renamed from its alias meaning and a numbered one is added; fold_instruction dispatches both, so a site that lacks the two appends fails to compile. The recorded template text still spells %i and %Ni, so nothing about the template changes across the version bump; the operand record grows two head words for the third list's count and token count, and the index goes to v26. Deriving %i at exec and index time from the command's OrderOnly edges was rejected. Edges keep written order and duplicates, but they cannot tell the rule's own list from a bang macro's (tup spells only the rule's: a macro whose `| y.h` joins a rule written `| z.h` renders i=[z.h]) and they carry no token grouping for %Ni. %f is recorded as a TokenList rather than derived from Normal edges for the same reasons. Order-only edges still come from both lists. The rule's own file operands resolve once now, feeding both the operand list and the edge; only the macro's paths go through the old edge loop. any_dep_changed is untouched: the edge walk already covers order-only dependencies, and a %i whose list changed is caught by the signature, which folds the rendered command exactly as it does for %f. Measured against tup on thirty-four rules, all agreeing after the change. %i spells the rule's list in written order, a group as <grp> or sub/<grp>, a bin as its members, a glob as its sorted matches, duplicates kept, a parent-directory file as ../z.h and a generated header by the spelling %f already uses; a bang macro's own order-only inputs and a group written in the inputs section are not in it. %1i and %2i pick tokens, %2i past the end and %1i over an empty list expand to nothing, %i and %1i in an output or extra output name are refused, both are allowed in the display string. A null rebuild after the first build is a no-op, so the recorded list round-trips. The review measured two pre-existing siblings. An unknown {bin} in an input or order-only list expands to nothing here where tup refuses the Tupfile with `Unable to find bin`, so `| {typo}` now trips the empty list refusal with a message tup never gives (#472). A dir/<grp> reference written from a nested Tupfile is spelled project-relative and its operand resolves to a group node with the directory joined twice, for %f on main and so for %i here; the corpus measured groups at the root Tupfile only, where both agree (#473). A glob-matched order-only input whose name holds a glob metacharacter gets no edge from the macro edge loop's text test; the rule's own list now gets its edges from operand kinds and so no longer has the gap (#474). Parsing examples/bsp costs 1.1% more instructions under cachegrind (1,096.8M to 1,108.9M) and a dry run 1.2% more; the dry-run command list is unchanged. Part of it is the third list itself, part is that a rule's order-only group patterns are now normalized both by the group pre-resolution loop and by expand_inputs, which the old code skipped for typed groups. Ref: #461 Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
PR metricsPerformance (gcc example, Linux)
Deterministic signals: instructions (cachegrind-simulated instruction reads — exact across runs, no PMU needed), page faults, peak RSS, and the cachegrind D1/LL miss rates. CPU time is user+sys from time(1). Internal statistics (gcc example, up-to-date dry run)
Counters from Binary size (Linux)
Code churn (whole codebase, last 30d)
Of the lines written across the codebase in the last 30 days, how many are already gone — work that was written and then discarded or rewritten inside the same window. This is the state of the tree including this PR, not a measure of the PR itself. Only code we write is counted: tests, examples, vendored and generated files, CI plumbing and prose are excluded. 963 lines were deleted in the window in total, most of them older than it. Where the churn is
Test coverage (lines)
105 files · 17802/20057 lines covered Deltas vs main@cfde6ad57. Updated for 1f70aed |
typeless
added a commit
that referenced
this pull request
Sep 19, 2026
Every `//` comment that is not a `///` doc header, a license line or a
NOLINT directive is removed from the seventeen C++ files PR #475
changed, together with the `} // namespace` closers and one `/*
done */` parameter name. The change is comment-only: with comments
and whitespace stripped, every file tokenizes identically to main
078387d5c. The other 168 files that still close namespaces with a
marker are untouched; they follow in later rounds.
The rule for this tree is that a non-obvious why lives in the commit
body or DESIGN.md, never in a comment, so what the deleted comments
said that the code cannot say is kept below, grouped by file and site,
for anyone who reaches this commit through blame. Plain narration and
restatements of the code were dropped without record.
src/graph/builder.cpp:
- resolve_input_node: Input paths are already source-relative from
expand_inputs() which normalizes them by combining with current_dir.
No further normalization needed here.
- resolve_input_node: For variant builds, paths like
"build/include/header.h" (from $(B)/include/header.h) already have the
build root prefix. Strip it to get source-relative paths.
- resolve_input_node: A seeded match names a file a later directory will
generate; without this it would resolve to a source-side Ghost and the
producer's output would become a second node.
- resolve_input_node: With BUILD_ROOT_ID model: source files are under
SOURCE_ROOT_ID (0) at source-relative paths; Generated/Ghost files are
under BUILD_ROOT_ID at source-relative paths.
- resolve_input_node: If path had build prefix, it's referencing a
generated file. Even if a source file exists at the same path, create
a Ghost node under BUILD_ROOT_ID so it can be upgraded to Generated
when the output rule is processed.
- resolve_input_node: In 3-tree builds, files may live in config_root
(alongside Tupfiles) rather than source_root. Check config_root as a
fallback for source file resolution.
- (file scope, §4 header): Path resolution uses graph traversal instead
of string manipulation (".." walks to the parent node, "name"
finds/creates a child). This naturally unifies input and output path
resolution because both traverse to the same node when the paths are
equivalent (e.g. $(B)/include/header.h from an input and the variant-
mapped output both resolve to the same node).
- get_or_create_group_node: Group nodes are stored with angle-bracket
name like "<gen-headers>".
- create_command_node: Sticky edges from condition env variables are
symmetric to condition_config_vars — so an env-driven guard flip
changes every guarded command's identity.
- create_command_node: An exported var reaches the command's subprocess
environment (read as bare $VAR) and so affects its output even when it
never appears in the command text. Recording it as a sticky edge makes
that dependency explicit and folds its value into the command
identity, so a change to the var triggers a rebuild. (process_export
guarantees every exported var has a value node.)
- expand_glob_pattern: Unconditional: the generated half of the match
set would otherwise depend on how far the parse fixpoint has
progressed.
- expand_glob_pattern: A file this round has not reached yet, but a
previous round proved the project generates. Without it the match set
is a function of where the parse has got to.
- expand_glob_pattern: Sorting and deduping across two path spaces would
order by where the build tree lives.
- apply_exclusions: Matched against the merged list, not re-expanded
against disk: generated matches are not on disk.
- expand_outputs: Guards do not gate this, and an absolute path is
rejected rather than re-rooted (#385).
- (file scope, §8): Forward declaration for mutual recursion:
include_single_file -> process_statement -> process_conditional ->
process_statement.
- include_single_file: A file joins each conditional context at most
once: the same include in the same guard stack is redundant, but a
later include under different guards is a distinct contribution (issue
#91).
- process_export: Per tup manual: "adds the environment variable
VARIABLE to the export list for future :-rules".
- process_export: An exported var reaches the command's subprocess
environment, so its value is part of the command's identity even when
the Tupfile never reads it.
- process_import: Per tup manual: "sets a variable inside the Tupfile
that has the value of the environment variable".
- process_import: Need null-terminated string for getenv.
- process_import: Resolution order is (1) environment; (2) if the author
wrote `?= default`, that default is the explicit revert-target for the
env-unset case and wins over the cache; (3) no env, no default — fall
back to the cached value from the previous build so plain `import VAR`
stays stable across shell sessions that forget to re-export the
variable.
- process_assignment: SortedIdVec moved-from state is empty, so no
explicit clear() needed.
- process_assignment: Capture the dependencies from RHS expansion. The
value also depends on every condition guarding this assignment:
reaching it required those config/env reads.
- process_assignment: Config variables are read-only (loaded from
tup.config), so only Regular and Node are writable.
- process_conditional: Env vars are tracked symmetrically with config
vars so that an env-driven guard flip (e.g. `ifeq
($(TUP_PLATFORM),win32)`) folds into the identity of every command in
the conditional block.
- process_conditional: Guards exist only for conditions that can flip
without a Tupfile edit (config or env reads recorded): static
conditionals are textual like tup.
- process_conditional: Macro defs and assignments in an inactive branch
serve only that branch's guarded rules; the active world after endif
must not see them.
- process_generated_rules: Order-only file edges are created from paths
with group refs already filtered out upstream.
- process_generated_rules: Edge from generated command to parent command
exists because dep-scan runs before compile.
- process_generated_rules: outputs intentionally left empty: generated
rules are dep-scan commands whose output is captured via
generated_output, not %o expansion.
- expand_rule: Early macro lookup is needed to process the macro's
order_only_inputs for demand-driven parsing.
- expand_rule: Effective-field merge of rule + macro: command/display —
macro wins (it IS the command template); outputs — macro fills in only
if the rule has none; groups — rule wins, macro is fallback; order-
only inputs — concatenated.
- expand_rule: Extra outputs are unioned, not fallback: a macro's extra
outputs are side effects its caller cannot restate (tup
parse_bang_rule_internal).
- expand_rule: Pre-resolve order-only group references so %<group> can
expand them in commands; this handles cross-directory groups like `|
../include/<gen-headers> |> cat %<gen-headers>`. Stores known group
names (sorted); the resolver constructs %<name> on the fly.
- expand_rule: Groups are first-class nodes; edges are created after all
Tupfiles are parsed.
- expand_rule: In tup, <group> references are always order-only even
when written in the inputs section.
- expand_rule: resolve_order_only_group is overridden for this rule's
command expansion so all %<group> patterns are preserved literally for
deferred resolution; the ScopeGuard restores it even on early returns.
- expand_rule: Uses scanner_registry (modular approach) if available,
falls back to pattern_registry.
- expand_rule: An extra output is owned exactly like a primary, minus
the %o operands and {group} (tup.1 extra-outputs).
- expand_rule: Generated means "produced by a rule this configuration
runs" (#386).
- expand_rule: Multiple commands may produce the same output when they
have mutually exclusive guards (same condition, opposite polarity) —
the phi-node case.
- expand_rule: Order-only edges from pre-expanded paths skip group
references (deferred edge creation) and glob patterns (not valid
paths).
- process_rule: Pending weak assignments (??=) are applied before
expanding commands so ??= assignments that precede rules take effect.
- process_rule: Skip rules where the input pattern evaluated to empty
(tup behavior): rule.inputs.empty() means no input pattern was
specified (": |> cmd"), inputs->empty() means the pattern(s) evaluated
to no files; only skip if a pattern was specified but produced
nothing.
- add_tupfile: In 3-tree builds config_root differs from source_root,
but the directory structure mirrors the source tree, so current_dir
relative to config_root is used for both config lookup and source file
glob expansion.
- add_tupfile: For 3-tree builds the Tupfile node is stored relative to
config_root (the Tupfile's actual location).
- add_tupfile: Config Variable nodes are created only once (first
Tupfile); subsequent Tupfiles reuse existing nodes.
- add_tupfile: The virtual $ directory for imported env vars mirrors
tup's env_dt; initialized only once, subsequent Tupfiles reuse
existing nodes.
- add_tupfile: Toolchain fingerprint: CONFIG_TRACKED_TOOLS lists tools
the build's outputs depend on that appear in no rule's inputs. Their
resolved (path, size, mtime) fold into every command's identity, so an
in-place tool upgrade rebuilds.
- add_tupfile: The env-var-used callback also ensures an env Variable
node exists for any env var that resolves from the environment, so
TUP_PLATFORM/TUP_ARCH (auto-resolved, not via `import`) get the same
tracking node as explicitly imported vars and fold into command
identity.
- add_tupfile: The resolve_order_only_group callback is for local group
references (no directory prefix) and uses the current directory;
groups are first-class nodes, looked up via graph edges (file ->
group).
- finalize_graph: Pass 1 accumulates members per (command, group_name)
keyed by packed (command_id << 32 | interned_group_name); same-named
groups from different directories contribute to the same replacement.
- finalize_graph: The configure pass runs before tup.config exists, so
rendered text does not yet distinguish anything and a key collision
there means nothing; it schedules only the config-generating rules
anyway. Last, because pass 2 rewrites command text.
src/cli/cmd_build.cpp:
- collect_scope_crossing_inputs: Derived from link_role rather than
enumerated here, so a new link type joins this bypass by being
classified once instead of by every walk remembering it — the omission
that made a declared source input invisible to a scoped build (#200,
#189). Ordering is excluded by the role, not by hand: order-only means
existence, not content.
- find_changed_files_with_implicit: A ghost recorded with no content is
one whose file was absent when the record was written, not one nothing
watches: its appearance is a change like any other (#386).
- find_changed_files_with_implicit: Excluded dirs are not parsed this
run, so nothing can consume their changes; observing them would record
state we have no authority over.
- find_changed_files_with_implicit: Past every skip above: this is the
one place that decides the walk looked at a file, so it is the one
place that can say so (#288).
- find_changed_files_with_implicit: Overlay: parse-time inputs live
under config_root but are indexed source-relative.
- find_changed_files_with_implicit: No active rule produces this, so its
*absence* is not a change; an appearance is, which is why this skip is
reached only once the file is known to be gone.
- find_changed_files_with_implicit: Accounted for already: deleted and
routed (#213), or read absent by its reader (#281).
- create_implicit_file: The run that reported this dep already saw it
absent, so only its return is news (#281).
- serialize_graph_nodes: Dropping a slot would shift every later load-
derived id (id == position + 1).
- serialize_graph_nodes: Overlay: parse-time inputs live under
config_root but are indexed source-relative.
- serialize_graph_nodes: Records that this build already routed the
file's absence, so the next one does not read the same stat failure as
news. The slot has to stay: id == position + 1.
- serialize_graph_nodes: A ghost that exists on disk but is produced by
no rule is a foreign input (e.g. the variant's tup.config): record its
content so change detection can see it.
- serialize_graph_nodes: An out-of-authority output arrives here as a
ghost; the record keeps its owner (#369).
- serialize_graph_nodes: Like every other file-shaped arm: one path, one
entry -- a discovered dependency naming this path must join this entry
rather than mint a second one.
- serialize_graph_nodes: These node types must be in index to maintain
consecutive ID sequence
- process_implicit_deps: Only when nothing orders this command after the
file. Ordering is transitive — a codegen emitting one declared and one
discovered output orders its consumer through the declared one — so
this asks reachability, not adjacency: enumerating path shapes is how
the two previous attempts at this taxed correct builds.
- preserve_old_implicit_edges: Command ids are positional and shift
across builds (e.g. when an earlier-created command is removed), so
the old edge's `to` id cannot be trusted to mean the same command; re-
resolve each carried edge's command through the same join every other
consumer uses.
- preserve_old_implicit_edges: If the command is gone, drop the edge. If
it survived and ran, the branch below drops it too: whatever that run
reported is now the whole truth, empty included.
- is_dir_authoritative: A dir under a pruned nested-project root has a
Tupfile this run never saw; its absence from available_dirs does not
mean it was deleted.
- merge_out_of_scope_commands: Content that moved, not content that
vanished (#247): the record stops claiming currency, not ownership.
- merge_out_of_scope_commands: Sole witness for non-operand inputs
(order-only, group, implicit, sticky).
- merge_out_of_scope_commands: An extra output is owned by its edge
alone, so the operand loop above cannot see it (#370).
- merge_out_of_scope_commands: Marked, not dropped: dropping retracts
which outputs this command owns along with the claim that they are
current, and only the second is in doubt (#241).
- merge_out_of_scope_commands: Omitted, not dropped (#243): an
unresolvable operand is one this record cannot describe, but the
outputs that did resolve are still owned by nothing else.
- merge_out_of_scope_commands: An operand it could not carry means the
record no longer describes what ran, so it keeps its outputs but stops
claiming they are current.
- merge_out_of_scope_commands: Implicit edges are carried by
preserve_old_implicit_edges instead.
- propagate_command_effect: Set before push_path decides: a caller that
dedups an already-changed path has still found an output, and forcing
the command as well would be wrong.
- build_index: After the discovered deps, so a merge-created copy of an
old entry cannot shadow the fresh one this build just stat'd — a
carried NodeFlags::AbsenceRouted would then discharge a later real
deletion as already routed, and the consumer would never run (#237).
Still before preserve_old_implicit_edges, which re-attaches carried
edges by identity and so must see the merged records; that, not the
ordering against the deps, is the constraint.
- reject_shadowed_sources: Only in-tree builds can destroy anything, now
that no output may leave the build hierarchy (#385): out-of-tree, %o
resolves under the build root, so a rule whose output path collides
with a committed file writes beside it rather than over it. The
collision is still confusing there -- the source becomes unreadable
through that path -- but it is not data loss, and rejecting on it
fails builds that cannot hurt anyone (a second build dir sees the
first one's artifacts).
- reject_shadowed_sources: What the previous build's record settles.
`Known`: a path it recorded as a source File is a source whatever is
on disk now, and a path it recorded while producing nothing at it is
owned by nobody (#389); absence stays undecidable -- a generated file
it does not mention is what a scoped build leaves behind for out-of-
scope outputs -- which is why the test below is positive.
`NeverBuilt`: nothing we produced can be on disk yet, so anything
sitting at an output's path is a source. `Lost`: a build happened here
and putup cannot tell either way, which is the only case that says so
out loud.
- reject_shadowed_sources: Guard-satisfied producers only, like every
other command walk here: an inactive conditional branch declares
outputs it will never write, and rejecting on those fails projects
that build fine.
- reject_shadowed_sources: The two-stage configure design has a rule
produce the tup.config that the same build then reads as
configuration. Match the whole basename: a suffix test also exempts
anything merely ending in those characters, e.g. mytup.config.
- reject_shadowed_sources: On disk in either input tree. Without this
the printed remedy -- delete the file and try again -- would not clear
the error, because the index still records it.
- reject_unresolved_ghosts: Guard-satisfied consumers only, like every
other command walk here (#386).
- reject_unresolved_ghosts: -a only pulls in rules this build left out
of scope; when nothing was left out, offering it sends the user round
the same failure (#222).
- detect_new_commands: A record that is not evidence outranks the
signature: the command must run again even when nothing about it
changed, and its outputs must be treated as changed so consumers of
whatever it half-wrote, or never wrote, are rescheduled too.
- reconcile_input_set: Every typed path, so "left the graph" below means
gone, not merely not-a-source: a generated file the user deleted is
still a graph node and must not read as removed.
- reconcile_input_set: Explicit comparator: libc++ extern-templates
std::__sort for unsigned int*, so the default form links against a
libc++ we deliberately do not have.
- remove_stale_outputs: Only delete outputs of a command whose directory
we have authoritative knowledge of this run; anything else is
preserved.
- remove_stale_outputs: Staleness is per file, not per command: a rule
that drops one of its outputs still joins through the ones it kept, so
asking only "did this command survive" would leave the dropped file
owned by nothing and never delete it.
- remove_stale_outputs: Not graph::command_label: the command has left
the graph, but the index keeps its operands, so a pattern shared by a
foreach still names one command.
- build_single_variant: Only scope parsing when explicit targets are
given. CWD-derived scoping should still parse all Tupfiles so that
out-of-scope Tupfile changes are detected for incremental builds. When
-a is set, always parse all Tupfiles so that cross-directory producers
are discovered and ghost nodes get resolved.
- build_single_variant: build_context loads the old index as part of its
work; that span is timed separately, so discount it here to keep the
phases disjoint.
- build_single_variant: No early exit on an empty graph: having nothing
to run is not having nothing to clean up (#231).
- build_single_variant: One line, not one per rule: a warning that fires
on every rule of a green build teaches everyone to scroll past
warnings. The per-object findings live in `parse`.
- build_single_variant: A record too old for read_index still says which
paths it recorded as sources; the version gate retracts its currency,
not that (#291). Nothing recovered this way is ever loaded as
`old_idx_ptr` -- it answers this one question and dies here.
- build_single_variant: Carries "has not succeeded since it last
failed": seeded from the previous index, cleared only by a successful
run. A build in which the command does not run at all -- a target or
scoped build -- must not forget it.
- build_single_variant: What the comparison below looked at, and so the
only files this build may restate (#288).
- build_single_variant: Outlives the block only so the ordering can be
derived after the up-to-date exit below.
- build_single_variant: Build the identity -> NodeId map: the cross-
build join key for commands. Must happen after parsing (operands set)
but before incremental logic.
- build_single_variant: Always include implicit deps (headers from .d
files) for in-scope commands, even if the headers live outside the
scoped directories.
- build_single_variant: Before detection, not after: a deleted output is
a change like any other, and a consumer reaching it order-only is only
notified if detection sees it gone.
- build_single_variant: A retired record is not "nothing to do": only
the write below persists the retirement (#245).
- build_single_variant: The command, not the display: "CC main.o" does
not say which flags broke, and this is the only line a build prints of
what actually ran.
- build_single_variant: Recorded even when it discovered nothing: an
empty report is a report, and treating it as silence carried dead
edges forever (#224).
- build_single_variant: Dropping a dep loses an edge no later build re-
derives, so neither arm is gated on -v.
- build_single_variant: Recorded, not ignored: tup drops out-of-tree
deps unless --full-deps, putup tracks them (DESIGN.md).
- build_single_variant: Nothing here will ever run a config rule, so a
needing-to-run record on one is undischargeable.
- build_single_variant: Past the up-to-date exit, not with the rest of
the incremental work: this is one pair per recorded discovery, and a
build that schedules nothing would derive all of them to run none.
- build_single_variant: Same pairs the scheduler gets, but kept on
contradiction: over-routing costs a run, not the build.
- build_single_variant: A command this build means to run is unverified
until it succeeds: an abort strands or kills it silently (#304).
- build_single_variant: Nothing here will ever run a config rule, so a
mark on one is undischargeable (as above).
- build_single_variant: Ahead of the dry-run branch so that a scheduler
which ever fails a dry job reports it.
- build_single_variant: "Would run", matching clean's "Would remove": a
dry run has completed nothing.
- build_single_variant: Persisting a partial failure is safe because
must_rerun records it, not because a failed command's outputs are
missing.
- build_single_variant: A build owes a record of what it ran: without
one the next build cannot tell it happened.
src/graph/dag.cpp:
- make_graph: Reserve BUILD_ROOT_ID (1) for the build root node. All
Generated/Ghost nodes will be parented under this node. The build
root's filesystem location is determined at build time (source_root
for in-tree, output_root for variant builds).
- make_graph: Index 0 unused, index 1 = build root
- make_graph: Name set by set_build_root_name()
- ensure_file_node: In-tree builds share one physical tree: a BuildRoot-
grounded path may already have a SourceRoot node (consumer parsed
before producer sees the on-disk output as a source file). Alias
instead of splitting.
- ensure_file_node: Lazy resolution: ungrounded PathIds are grounded
before creation. Try BuildRoot first (outputs are more common), then
SourceRoot.
- set_build_root_name: path_id stays PathId::BuildRoot (set in
make_graph). The name is for display only (get_full_path), not for
PathId identity.
- compute_command_key: Command text is Tupfile-relative, so the same
rule in sibling directories renders identically; without the directory
those distinct rules share one key.
- compute_command_key: A dep-scan command is output-less and drops its
parent's -o, so two compiles of one source with equal flags render
byte-identical scans; whose deps they inject is the only thing that
tells them apart. One level: a parent is rule-authored, so has none.
- compute_command_signature: An output the text never names -- every
extra output, a %o-less primary -- would otherwise leave identity
unchanged when the declaration changes; sorted for insertion
independence. A dep-scan command's data-flow edge to its parent shares
this walk and names no file.
- compute_command_signature: Fold in (name, value-hash) of each Variable
node reached via a Sticky edge. This captures vars that affect output
without appearing in the rendered text — exported env vars the
subprocess reads as $VAR, config vars gating an export, etc. Sorted
and deduped by name so identity is independent of edge insertion order
and tolerant of duplicate sticky edges (the graph permits them).
- compute_affected_commands: A contradictory pair only adds a command
here: the set only grows, so no cycle check.
- compute_affected_commands: Joined after the cascade, not before: a
forced command runs, but nothing about it says its consumers must, and
seeding it upstream would say exactly that.
- compute_affected_commands: InjectImplicitDeps siblings (dep-scan
commands) have no graph outputs, so the cascade above can't reach
them. They must run whenever their parent compile runs — otherwise
newly-introduced transitive includes are never re-discovered, and the
parent's run drops the recorded ones with nothing to re-report them
(#228). Walk all commands and attach any whose parent_command was just
marked affected.
src/parser/eval.cpp:
- lookup_var_with_bank: Context-computed special variables (highest
priority) - NOT overridable
- lookup_var_with_bank: TUP_PLATFORM and TUP_ARCH: env > config >
context > default
- lookup_var_with_bank: Fall back to config (tup behavior: CONFIG_*
accessible via $())
- expand_var: Dependency tracking based on which bank was used. An @()
reference is a config read even when the variable is undefined —
defining it later must count as a change.
- expand_var: TUP_PLATFORM/TUP_ARCH fall back to compiled-in defaults
when neither env nor config provides them; setting the env var later
must count as a change, so the fallback still records an env read.
- expand(EvalContext&, Expression const&): Recursively expand any
variable references that were embedded in literals (e.g., from escaped
quotes like \"$(VAR)\")
- expand_path: Split result by whitespace - variables may contain
multiple files
src/index/reader.cpp:
- declared_layout_fits: Operand data is the only section whose size the
header does not carry: it runs from its own offset to the string
table.
- open_index_in_window: Damage before version: neither a sub-header file
nor foreign magic is a record of any version (#383).
- open_index_in_window: What makes the bytes worth reading at all: that
they are the ones a putup wrote, not a file that merely starts with
the right four and lays its sections out plausibly (#294).
- read_index: A field this read cannot reproduce faithfully makes the
whole record unreadable rather than a weaker claim in its place: an
empty name or operand set is something callers act on (#381).
- prior_paths: Total by construction: a reader that decides whether to
overwrite a file must not consume a projection that drops an entry
type silently (#389).
- prior_paths: Each entry feeds one list, so a path appearing twice --
in one list or across two -- is a record holding two entries for one
path, the state its own writer forbids, and this is the last check
available before an irreversible action (#382). Compared as stored: an
out-of-tree record legitimately holds one file as "a/bar.txt" and
"build/a/bar.txt", which stripping a prefix would collapse.
- read_prior_paths: The file table and nothing else. Everything a
version bump invalidates -- command identities, signatures, recorded
currency -- stays unread, so it cannot reach a caller through this
return type.
- index_get_semantic_string: The string table's own end, not the file's:
a string declared past the table but inside the file is still a string
this record does not contain.
- index_get_semantic_string: Offset 0 is the table's empty entry, so
this is a value the record states, not a failure.
- index_get_operands: Widened like every position here: in u32 this sum
wraps, and the check below then passes (#372). The operand section's
own end, like the string table's: the writer lays the string table
directly after this section, so a record declared past it is one this
record does not hold.
src/index/writer.cpp:
- serialize_index: Load derives ids from position, so a gap silently
rewires every later edge.
- serialize_index (add_display): Identity is key/signature, not this
text (#360): mark and shorten rather than fail the record.
- serialize_index (command_entries loop): v8: Use instruction_pattern
instead of fully-expanded command.
- serialize_index (push_u32): The 4GB guard below is the only bound on
these, and only while each counted operand costs >=1 byte of this
stream (#365).
- serialize_index (edge sort): Canonical order: edge bytes must not
depend on job-completion order (#120).
- serialize_index (save_time_ns): Get current time for racy-clean
detection.
src/index/entry.cpp:
- Index::compute_paths: File IDs are 1-based contiguous: files_[i].id ==
i + 1. Use direct indexing instead of a hash map.
- Index::compute_paths: Resolve paths top-down (chain is bottom-up, so
iterate in reverse)
include/pup/graph/dag.hpp:
- CommandNode::guards: Condition guards - command executes only if ALL
guards are satisfied
- CommandNode::guards: For nested conditionals, this accumulates all
enclosing conditions
- Graph::paths: mutable: memoized interning cache, written on the
(const) read path. Single-threaded access only — not safe to share
across threads.
- BuildGraph::path_cache: mutable: memoization written by get_full_path
on the (const) read path. Single-threaded access only — not safe to
share across threads.
include/pup/index/format.hpp:
- RawCommandEntry (static_assert on sizeof(RawCommandEntry) == 88):
Widening this record adds a category of recorded state, and a category
is only useful once all three of its legs exist: written here,
compared where staleness is decided, and routed so the commands that
depend on it are scheduled. A category with a missing leg is a silent
wrong build, which is the shape #189 catalogues — so the size is fixed
deliberately, to stop a new field reaching the index before someone
has answered for all three.
include/pup/index/entry.hpp:
- Index (save_time_ns_ field): Index save time (nanoseconds since epoch)
for racy-clean detection
test/unit/test_e2e.cpp:
- is_noop accepts only a build that ran nothing: The assertion most
incremental scenarios rest on, so it has to be exact: a substring test
for "0 commands" also accepts every multiple of ten, and a quiescence
check that can pass while a build ran hides the very defects those
scenarios exist to catch (#234).
- An unterminated caret is a parse error, not a shell error: Upstream
rejects this at parse time; falling through left the ^ in the command
text, so the rule died mid-build as "sh: ^: not found" — blamed on the
tool, not the typo (#217).
- An upstream caret flag is rejected, not rendered as a label: tup reads
the non-space run after ^ as flags (t, o); putup implements neither,
and printing "t" as the rule's label honours nothing and refuses
nothing (#217).
- A failing command is reported by its command line, not its display:
The display names the step, not what broke, and this is the only line
a build prints of what actually ran.
- A bang macro's display wins over one written on the rule: Outputs and
groups resolve rule-over-macro; display is the one field that goes the
other way, because the macro owns the command the display names.
- Bang macro order-only groups trigger demand-driven parsing: This test
verifies that order-only group references embedded in bang macros
correctly trigger demand-driven parsing of the directory containing
the group. Bug: the group reference in !cc = | $(TOROOT)/include/<gen-
headers> |> ... was not triggering parsing of include/Tupfile before
looking up the group.
- Bang macro order-only groups work in variant builds: This test
verifies that order-only group references in bang macros work
correctly in variant builds. The bug was that DEP (implicit dep
scanning) commands were not inheriting the order-only edges from their
parent compile commands when using bang macros with TOROOT-based group
references like: !cc = | $(TOROOT)/include/<gen-headers> |> ... This
caused DEP commands to run before headers were generated.
- Group references in regular inputs expand correctly: This tests the
spos pattern: $(ROOT)/modules/<json-headers> |> cat %<json-headers>
Group references are order-only even when in regular inputs section
- A Tupfile edit that changes no command re-runs nothing: Complement of
"Editing an output-less command re-runs it". The rule needs an output:
the Sticky route propagates a command's outputs, so an output-less one
cannot exhibit this at all (#225).
- Touch does not trigger unnecessary rebuild: Use absolute path to touch
since it's not in workdir
- A build record that is not putup's own is refused out loud: Mid-file:
past the header putup already validates, ahead of the footer, so
nothing but the checksum can notice.
- A build record that is not putup's own is refused out loud: With the
record refused, nothing left says which files on disk this project
produced, and #291's rule is that putup does not guess -- so the build
stops and names them rather than treating its own outputs as checked-
in sources.
- A build records one entry per path: The directory walk creates an
entry and registers it for the next lookup; register anything but what
it created and the next chain re-creates it (#325).
- A dependency outside the source tree is recorded rather than dropped:
The arms that drop a discovered dep sit above an else that keeps the
out-of-tree ones as absolute paths. Nothing pinned that, and #305 was
filed on the assumption it drops them.
- A changed header re-runs the output-less command that read it: Routing
a discovered dep pushes the reading command's outputs, so a command
with none was reached and then dropped. A compile gate is the shape
that has no outputs on purpose (#228).
- A scoped build sees a declared input outside the scope change:
Detection skips out-of-scope files unless a bypass covers them, and
the bypass admitted Implicit and Sticky edges but not Normal — so a
plainly declared source input was the one kind of dependency a scoped
build could not see (#200).
- A command that raced its discovered dependency runs again: Nothing
orders a consumer against a producer it only discovers, so it can read
the file before it exists. The dep is then recorded from a post-run
stat — as already satisfied — and no later build re-runs it, leaving
the output permanently wrong (#274).
- A consumer ordered through a sibling output is not taxed for
discovering the other: Codegen emitting one declared and one
discovered output: the consumer is ordered through the declared one,
so it cannot have raced the discovered one. Marking it anyway doubles
every codegen rebuild — the shape a two-hop ordering check cannot see
(#274).
- A discovered dependency orders its consumer on a later build: A
discovery is index-only, so it orders nothing: every later build that
runs both the producer and the consumer races them again, and the
consumer is taxed with an extra run to catch up. The previous build's
discovery is what the scheduler orders by now (#276).
- A recorded discovery that the rules now contradict does not stall the
build: The ordering carried from the last build is stale by
construction: the rules can since have turned the discovered file's
producer into a consumer of the discoverer's output. The rules are
this build's truth, so the contradiction retracts the carried ordering
— taking it as binding leaves both commands waiting for each other,
and a scheduler with nothing runnable and nothing running reports the
build complete having run neither (#276).
- A discovery whose producing rule is gone orders nothing: Ordering
carried from the last build names a producer by the file it produces,
so a rule that has since stopped producing it names nothing and the
consumer waits for no one (#276).
- Ordering the scheduler did not enforce does not excuse a race: A
carried ordering is only real for a pair this build actually
scheduled: the consumer may be quiescent and absent from the job set,
and then nothing enforced it. Crediting it anyway lets a command reach
its own excuse through the missing one's outputs, and the race it did
commit goes unmarked — the permanently wrong output #274 exists to
prevent (#276).
- A producer's own input change reaches the consumer that only
discovered it: The affected cascade walks graph edges from the pre-
build changed set, and a discovered dependency has none; the file is
not known changed until its producer has run, and the index then
stamps it from a post-run stat. So the consumer was never scheduled at
all and its output stayed wrong while the build reported the tree up
to date (#277).
- A producer's input change reaches a chain of discovered consumers:
Routing a discovered consumer makes its own outputs change, so
whatever discovered those must follow. Expanding the recorded pairs
once rather than inside the cascade's fixpoint would reach the first
consumer and stop (#277).
- A producer's input change reaches an output-less discovered consumer:
The cascade marks the consumer command node itself, so a reader with
no output path is reached the same way one with outputs is; routing
through outputs instead would drop it, the shape #228 had on the
comparison route (#284).
- A producer's input change reaches an output-less discovered consumer:
The gate's dep scan must see the header to record it; introduced
together they race and the discovery is silently empty.
- A discovered consumer re-runs for a producer that rewrites the same
bytes: Membership routes, not content: the consumer re-runs because
its producer ran, exactly as a declared consumer does. Pinned so the
pessimism is a decision rather than a surprise, and so that it costs
one run per producer run and not one per build (#277).
- A dependency absent when it was recorded settles rather than re-
running forever: A command may report reading a file that is not there
-- a conditional include that resolved to nothing. Reading the same
stat failure as news every build re-runs the command forever for
output that cannot change, which is the loop the campaign exists to
kill. tup records the absence as a ghost and settles, and re-runs only
if the file appears (#281).
- A dependency that was absent when recorded still re-runs its reader
when it appears: What settling must not cost: the absence is recorded
as zero size, zero mtime and a zero hash, and a file arriving has to
be read as a change against all three (#281).
- A dependency that was absent when recorded still re-runs its reader
when it appears: Size and mtime match the sentinel zeros, so only the
recorded hash separates it.
- A deleted dependency a command still reports re-runs it once: The
deletion is a real change and must reach the reader, but the run that
follows records the file as absent -- so the build after it has
nothing new to say and must settle (#281).
- A recreated dependency does not carry its deletion mark forward: The
merge copies an out-of-scope file's entry verbatim. Run before the
discovered deps, that copy shadowed the fresh one, so a carried
NodeFlags::AbsenceRouted discharged the next real deletion as "already
routed" and the consumer never ran again (#237).
- A recreated dependency does not carry its deletion mark forward:
Nothing orders these two commands on a first build — b/ depends on
a/p.txt only by discovery — so c.o's content here is whichever won,
and asserting it raced on CI. gen.sh writes the .d either way, so the
dependency is recorded regardless, which is all the steps below need.
- A build whose discovered dependency was deleted quiesces: The command
re-runs and rediscovers nothing, which the carry logic could not tell
from "did not run", so the dead edge and its file entry came back
every build (#224).
- Implicit deps survive identical rules in sibling directories: Command
text is Tupfile-relative, so these two rules render the same string.
If identity ignores the directory they collide, and one directory's
header edges get attached to the other's command.
- A header the compile reads only under -O2 is tracked: A scan without
the compile's flags resolves the other branch and records the wrong
header.
- A dep scan that prints anything but its rule fails the build: Chatter
parsed as dependencies never stats, so the command would re-run for
ever.
- Implicit deps survive a flag whose path is a separate word: A flag's
path reaches the scan whichever spelling carries it, or the scan has
no input file.
- Implicit deps cover every source of a multi-source command: gcc -M
emits one rule per source; stopping at the first leaves b.h untracked
and the program silently stale
- Implicit deps survive command-id shift from a removed source: Implicit
(header→command) edges discovered last build are carried forward for
commands that don't rebuild. If that carry-forward keys on the
command's array position (NodeId), removing a glob-matched source
whose command was created earlier shifts every later command's id down
— and the carried edge gets misattributed to whatever command now
occupies the old id. A later edit to the header then rebuilds the
wrong unit and leaves the real output stale. The carry-forward must
key on the command's structural identity, stable across shifts.
- Implicit deps survive command-id shift from a removed source: No
Tupfile edit: the glob drops m_b.c. m_a.c is not recompiled this
build, so its m_a.h dependency must be carried forward — at the new
command id.
- A build run from a subdirectory does not stamp an out-of-scope change
as current: A cwd-derived scope parses the whole project but detects
only its own directory, while the record leg re-hashed every graph
file: a/src.txt was recorded current on the strength of a stat nothing
consumed, so the next full build compared v2 against v2 forever
(#288).
- A build with --all-deps does not stamp an out-of-scope change as
current: -a empties parse_scopes for the same reason cwd scoping does,
so it reaches #288 by the same door: everything is parsed, only the
scope is detected, everything is recorded.
- A file added while building from a subdirectory is still built: The
other side of #288's fix: carrying an unexamined file's recorded state
forward must not become "record nothing", or a file first seen by a
scoped build would never be built.
- A stale output that cannot be deleted fails the build and keeps its
record: Not SKIP: the suite is built -fno-exceptions, where Catch2
aborts the process instead.
- clean does not count an empty directory it could not remove: The
platform message already names the path; the caller must not repeat
it.
- A source and the out-of-tree output shadowing it are one record clean
can read: The regression pin for comparing recorded paths as stored:
this record holds one file as both a source and an output, and only
their spellings tell them apart (#382).
- A build refuses to overwrite a file the record does not attribute to a
rule: Whether a previous build happened must not change the answer:
the record is what the guard reads, and none of these three sequences
gives it a claim on the file.
- Turning a branch off keeps ownership of what it built: Ownership
survives the branch going inactive only via the record's carry-forward
(#369).
- A stale output that cannot even be queried keeps its record: The rule
lives in the readable root Tupfile so its directory stays
authoritative; only the output's directory is locked, which is what
reaches the exists() guard.
- Source file content change triggers rebuild in variant build:
scoped_build has: app -> lib cross-directory dep app/Tupfile is parsed
first (alphabetically), references ../lib/foo.o This may create Ghost
nodes, testing the ID contiguity fix
- Source file content change triggers rebuild in variant build: Modify
"42" to "99" - same size (2 chars), different content
- Tupfile changes detected regardless of scope: A comment would not do:
an edit leaving every command's text identical is correctly a no-op
(#225), so the mutation has to change a command.
- A file that cannot be hashed is named in a warning: The only signal a
user gets when content hashing fails; it had no test at all (#204).
- A file that cannot be hashed is named in a warning: The command must
not read in.txt, or it fails before putup ever hashes it.
- Order-only deps on generated outputs resolve correctly in variants:
This tests the bug where order-only deps using plain paths (e.g.,
"include/foo.h") created duplicate File nodes instead of reusing
Generated nodes. Pattern from busybox: : |> ... |> include/applets.h #
output becomes build/include/applets.h : | include/applets.h |> ... #
should resolve to build/include/applets.h Bug: the order-only dep
created a File node at "include/applets.h" instead of reusing the
Generated node at "build/include/applets.h".
- Variant outputs are automatically mapped to build directory: This
tests that output paths are automatically mapped to the variant
directory. Pattern from busybox: Root Tupfile: : |> ... |>
include/header.h # should become build/include/header.h src/Tupfile: |
$(B)/include/header.h # references build/include/header.h Without
automatic mapping, the output creates a node at "include/header.h"
(source), but the dependency references "build/include/header.h"
(variant) - creating two nodes.
- Variant outputs are automatically mapped to build directory: If output
was mapped incorrectly, there would be a ghost node at
build/include/header.h that never gets satisfied
- Cross-directory regular inputs work in variant builds: Similar to
order-only test but with regular input dependency This tests that
Ghost->Generated upgrade preserves edges aaa_consumer is parsed first
(alphabetically), creates Ghost for ../zzz_producer/helper.c
zzz_producer is parsed later, upgrades Ghost to Generated The edge
from aaa_consumer's command to the generated file must be preserved
- Self-host test via shell fixture: Regression guard: when a source's
already-tracked header is edited to transitively include a new header,
pup must record the new transitive header and rebuild on subsequent
edits to it. Fixed by binding dep-scan command dirty-status to its
parent compile (collect_affected_commands).
- Subdir inherits from parent when no local config: NO
build/sub/deep/tup.config
- Root config used when no intermediate configs: NO build/sub/tup.config
- should inherit from root
- Empty subdir config does not block parent merge: Empty — parent vars
merge through
- Configure uses root tup.config only: NO build/configs/tup.config
- Configure does not create .pup directory: Do NOT call init - no .pup
directory exists
- Configure handles config rule depending on non-config rule: Bug:
passing only config-output commands as a filter ignores their
dependencies. If a config rule depends on an intermediate file
produced by a non-config rule, the dependency is not run, causing the
config rule to fail.
- A configure that cannot write tup.config does not report creating it:
A file where the directory must go beats the exists() guard on
tup.config itself, and needs no permission bits, so the scenario runs
under root too.
- Config selection persists across multiple builds: Use simple_c fixture
which has no config-generating rules
- (7 sites: permission/skip guards at former lines 3472, 3554, 3605,
3925, 4503, 8413, 8464): Not SKIP: the suite is built -fno-exceptions,
where Catch2 aborts the process instead.
- Scoped build must not delete out-of-scope outputs section: issue #122
- A build aborted before a command could run does not record it as done:
Self-validating: the point of the scenario is that an abort leaves
commands it meant to run un-run, so it must witness one before
asserting what follows.
- A build that cannot save its record does not report success: Renaming
onto a directory fails for root too, so this needs no permission bits
and no skip guard, unlike the scenarios that revoke write access to a
directory.
- A failed command's consumer runs once the command succeeds: The
consumer's output already exists holding V1, so nothing schedules the
consumer except propagation from the producer's re-run. Routing the
retry through forced_cmds instead of changed_outputs leaves final.txt
at V1 forever, which is what this pins.
- index_shape: Determinism of construction, not of reload: two builds
that never saw each other's work must record the same project.
Answering this by reading the code is what #298's consult had to do.
- Two builds of one tree record the same thing: A record whose sections
are identical but whose lengths are not would mean the masking above
is hiding a difference rather than excluding a timestamp; a shape that
covers only the header would mean it is comparing almost nothing.
- Two builds of one tree record the same thing: The weaker property the
incremental suite leans on, asserted here because it costs one build:
a run that does nothing must not rewrite what the record says either.
- A damaged record says so instead of rebuilding in silence: Without
this the shadow guard speaks first: an output on disk with no readable
record is the #291 refusal, which would pass this scenario for the
wrong reason.
- A record too short to hold a header says so instead of rebuilding in
silence: Below sizeof(RawHeader) + sizeof(RawFooter): above it the
declared-layout row answers first and this would pin the wrong
rejection.
- A record too short to hold a header says so instead of rebuilding in
silence: Without this the shadow guard speaks first, as in the layout
scenario above.
- Distcleaning keeps a record whose files it could not remove: The
removal failure names the file on its own, so only the keep message
witnesses that the record was kept because of it.
- A glob's %f order does not depend on the build directory's name: Pins
both halves and the canonical order: equality alone would still hold
if the filesystem half stopped contributing entirely.
- Out-of-tree, a generated file shadowing a source is one glob match and
the source survives: Out-of-tree the output lands under the build
root, so the committed file is shadowed rather than overwritten --
confusing, but not data loss, which is why #194's rejection is limited
to in-tree builds. What #191 guarantees here is that the file is one
match rather than two.
- A glob consumer of a deleted stale output settles after the healing
build: The heal in build 2 is #212 and is deliberately better than
tup, which runs the consumer zero times; only the third build is the
defect (#213).
- A rule still naming a deleted stale output is rejected rather than re-
run: The first rebuild is #212's routed heal; the defect is the build
after it, which re-detects the file putup itself deleted (#213).
- Removing a group member re-runs the commands that consume the group:
The rule vanishes with its glob match, never by a Tupfile edit: an
edit rebuilds a surviving member, and a rebuilt member reaches the
consumer over the live graph, masking the removal (#169).
- Removing a group member re-runs the commands that consume the group:
Names the consuming command too, which an unannotated rule did not
(#229).
- Removing a group member re-runs the commands that consume the group:
The removed foreach instance, not the pattern its sibling also
matches.
- Removing a group member schedules nothing when the group's only
consumer is guarded off: Edges into a guard-unsatisfied command are
dropped when the index is written, so the walk finding nothing here is
the correct answer and not a missed route.
- Env var change in a conditional rebuilds the affected branch: Both
branches of `ifeq ($(TUP_PLATFORM),...)` produce the same output file
with distinct content; the platform string does not appear in either
command's text. So the only signal that an env-driven branch flip
occurred is a sticky edge from the env Variable node to the guarded
commands (the condition_env_vars path, symmetric to
condition_config_vars). Without it, change detection sees no changed
file and no changed identity, reports "Nothing to do", and the output
stays stale.
- Imported env vars persist across builds: MY_VAR not in environment
(EnvGuard out of scope)
- Exported env var consumed via subprocess environment triggers rebuild:
The command consumes MY_GREETING through the inherited environment
(bare $VAR in the shell), NOT via $(VAR) substitution. So the rendered
command string is byte-identical across values: a string-keyed change
detector cannot see the change. Correctness requires the command's
identity to fold in the values of the vars it depends on (here, the
exported MY_GREETING).
- ?= soft assignment - = takes precedence: Use ?\?= in strings to avoid
trigraph interpretation (??= -> #)
- Cross-directory groups in 3-tree builds: Mirrors the GCC example
pattern: root Tuprules.tup: S = $(TUP_CWD); LIB_DIR = gcc
gcc/Tuprules.tup: S ?= $(TUP_CWD); LIB_DIR ?= .; macros use
$(S)/$(LIB_DIR)/<group> gcc/Tupfile: produces <gen-headers>, consumes
via macros
- Config tree inside source tree: main.c lives in source_dir, not
source_dir/tupfiles, so a rule for its object exists only if the glob
matched it there.
- Cross-project order-only dependency resolution: This tests the bug
where order-only deps using $(B) paths in 3-tree builds create Ghost
nodes instead of resolving to existing Generated nodes. Pattern from
busybox: root/Tupfile: : |> ... |> include/autoconf.h # output at
build/include/autoconf.h applets/Tupfile: : main.c |
$(B)/include/autoconf.h |> ... The bug manifests when: 1. source and
output are in completely different filesystem trees 2. build_root_name
has N levels of "../" (e.g., "../../../../tmp/build") 3. From a
subdirectory, $(B) normalizes to N-1 levels of "../" because one "../"
cancels with the subdirectory name 4. strip_build_prefix() fails to
match the different prefix depths Example with busybox: source =
/home/user/src/busybox output = /tmp/build build_root_name =
../../../../tmp/build (5 levels up from source) From applets/: $(B) =
TUP_VARIANT_OUTPUTDIR/.. = ../../../../tmp/build/applets/.. =
../../../tmp/build (4 levels)
strip_build_prefix("../../../tmp/build/include/x.h",
"../../../../tmp/build") FAILS - prefixes don't match due to depth
difference!
- Cross-project order-only dependency resolution: Create asymmetric
setup: source at 3 levels deep, output at 1 level source_root =
workdir/a/b/c/source (3 dirs deep) output_root = workdir/out (1 dir
deep) build_root_name = relative(out, a/b/c/source) = ../../../../out
(4 ../) From consumer/: $(B) expands and normalizes to ../../../out (3
../) The prefix mismatch causes strip_build_prefix to fail
- Sibling directory inputs work with incremental variant builds: This
tests the command string matching between graph and index. Pattern
from spos: Tupfile at include/generated/ referencing ../data.txt Bug:
Index uses std::filesystem::relative() while graph uses
make_source_relative() These produce different paths for cross-
directory references.
- The object of a compile that runs elsewhere is reported instead of
scanned wrongly: The scan runs from the Tupfile's directory, so a
source word taken from an invocation that ran in sub/ resolves against
a same-named file here -- deps recorded for a file the rule never
compiled (#356).
- The object of a compile that runs elsewhere is reported instead of
scanned wrongly: One is standing next to it: a.o's compile is the
covered prefix. The report's unit moved to the object, so its sentence
has to speak about the object.
- An object no scanned invocation writes is reported beside its scanned
sibling: %o one directory down expands to '../../build/src/lib/a.o' --
the word resolves against the rule's own directory and back into the
variant, which a root-level rule never shows.
- A compile-shaped rule with no dependency scan is reported: The scan is
declined correctly — putup cannot reproduce the prefix's shell state —
but declining in silence leaves a rule whose headers are never
recorded (#352).
- A depfile flag the compile never carried hides no unscanned object:
The suppression exists for a compile that writes its own depfile; read
across the whole command text it was defeated by any word spelling
one, including in a later invocation (#357).
- Scoped build detects header changes outside scope: Reproducer for the
real-world bug noted in CLAUDE.md: "Header-dep tracking can miss
across variants — editing a widely-included SDK header may re-link
with stale .o files and produce a binary newer than the header but
functionally older." Differs from the single-scope test above in two
ways: - multi-arg scoped invocation (mirrors `pup src/fubon ... ios`
pattern) - multiple TUs in different scope dirs all transitively reach
the same out-of-scope header, AND a downstream link rule consumes
their .o files The strict assertion checks that ALL three .o files
(alpha.o, beta.o, main.o) are present AND the binary's hash changes
after the header bump, catching both the "noop" mode and the "rebuild
some-but-not-all + relink with stale .o" mode of the bug.
- Scoped build with multiple scopes detects out-of-scope header changes:
Capture the linked binary's content so we can detect stale-link bugs
even if pup claims "rebuilt".
- Scoped build with multiple scopes detects out-of-scope header changes:
Strict check: the linked binary must differ from the pre-edit version.
If pup recompiled only some .o files and re-linked with stale others,
the binary would be byte-identical or only partially updated — either
way still buggy.
- Scoped build with multiple scopes detects out-of-scope header changes:
Same shape as above but with the build done out-of-tree under
build/<variant>, invoked with -B. Variants share the source tree but
have independent indices — a fix in one variant's index must not let
the other re-link with stale .o.
- Out-of-tree variant build picks up out-of-scope header changes: Probes
the scoped-initial-build axis: when the FIRST build is already scoped
(rather than full), is the index populated correctly enough that a
subsequent header edit propagates through the linked binary?
- Scoped rebuild after a SCOPED initial multi-scope build: Probes the
repeated-scoped-invocation axis: full build, then a no-op scoped pass,
then a real edit + scoped rebuild. Mirrors the production pattern
where developers run the same scoped command many times across a
session before the edit that exposes the bug.
- 3-tree: group pattern %o must include build root prefix: Reproduces
GCC BSP pattern where: 1. Library archive uses order-only group:
%<objs> in command 2. Consumer links the library via
$(B)/$(LIB_DIR)/libmath.a The group pattern forces
has_group_pattern=true, so final_instruction becomes cmd_text (parse-
time %o expansion). Output PathIds must be BuildRoot-grounded so
materialize_path() prepends the build root prefix. Without grounding,
%o becomes "libmath.a" (bare filename) instead of
"../../build/zzz_lib/libmath.a".
- 3-tree: group pattern %o must include build root prefix: The fixture's
display is "AR %o", so the word after it is the rendered %o: a bare
"libmath.a" is the bug, meaning the archive was written to the source
tree.
- 3-tree: group pattern %o must include build root prefix: Bounded to
the line: %o is the last word on it, and the next line begins
"[build]", which would satisfy the check below on its own.
- An output declared by both conditional branches stays tracked: Scoped
build must not delete out-of-scope outputs (issue #122)
- An output declared by both conditional branches stays tracked: A two-
directory project (alpha, beta) plus a root-level rule, fully built in
build/. Each rule just copies its source, so outputs are trivially
checkable.
- A glob over generated files is path-ordered and stable across builds:
The generated files exist on disk from here on, so a filesystem glob
can now see what only the graph could see during the first build.
- A continuation without a space before it builds and is scanned:
Upstream tup rewrites `\`+newline to spaces, so the command runs as
`gcc -c foo.c`; keeping the two bytes made the shell splice them into
`-cfoo.c` and hid the compile from the dep scanner.
- A group directory prefix after an extra outputs section names the
group: An unresolved group only warns, so the silence is what
witnesses the prefix moved it.
test/unit/test_index.cpp:
- `pup::graph::add_condition_node` forward declaration: Graph-module-
internal (defined in dag.cpp, not in public header)
- `TEST_CASE("CommandEntry conversion")`: Distinct values: a roundtrip
that swapped the two fields must fail.
- `TEST_CASE("Serialized edge section is independent of edge insertion
order")`: Ties on from (edges 4->cmd1 / 4->cmd2) and on (from, to)
(Implicit vs Sticky 4->cmd1) exercise every leg of the canonical-order
comparator.
- `TEST_CASE("Index ID contiguity requirement")`: This test documents a
design constraint: IDs must be contiguous when stored in the index.
The index format assigns IDs from array position on load (id =
array_index + 1), so if there are gaps in stored IDs, parent_id
references will be broken after round-trip. The build system ensures
ID contiguity by storing ALL node types (including Ghost, Variable,
Group) rather than skipping them. This test verifies the consequence
of violating this constraint.
- `TEST_CASE("A record whose declared layout does not fit the file is
refused")`: Reading it as empty was the old behaviour: an out-of-
bounds section became a zero-length span, so a damaged record loaded
as the record of a project that had produced nothing -- and the
source-file guard, which reads exactly that, went quiet and let a rule
overwrite a checked-in file (#293).
- `TEST_CASE("A file that does not carry the index magic is damage")`:
No re-signing: the magic check runs before the checksum, so this is
the arm under test.
- static_asserts before `TEST_CASE("An operand offset that wraps makes
the record unreadable")`: Both wrap tests discriminate only because
the position they wrap onto holds something: a wrap landing on zeroes
would be rejected for the wrong reason, not for the wrap the test is
about.
- `TEST_CASE("A name offset that wraps makes the record unreadable")`:
Planted in this entry's `size`, which nothing reads back, so the
wrapped read finds a name.
- `TEST_CASE("A record whose entry carries a type this putup cannot name
is unreadable")`: The recovery read builds the same entries, so it
must not answer for this record either.
- `TEST_CASE("A name offset that wraps makes the record unreadable")`:
The record that names it is unreadable, so the recovery read must not
answer for it either.
- `TEST_CASE("A record whose edge carries a link type this putup cannot
name is unreadable")`: Damage in a section the recovery read never
looks at leaves the recorded paths readable.
- `TEST_CASE("A record whose edge carries link type zero is
unreadable")`: LinkType starts at 1, so zero is as unnameable as
anything past the last enumerator.
- `TEST_CASE("An empty recorded string is a value rather than a
failure")`: env is semantics-bearing and empty here, which is exactly
the value a failure must not mimic.
- `TEST_CASE("A record names the paths it recorded even at a version too
old to trust")`: The window is a handful of integers, so it is
enumerated rather than sampled.
- `TEST_CASE("A record that classifies one path two ways is
unreadable")`: The three lists partition the file table, so one path
in two of them proves the record holds two entries for it -- a state
its own writer forbids (#382).
- `TEST_CASE("A record that names one path twice with one type is
unreadable")`: The same rule from inside a single list: two entries
for one path, whichever list…
typeless
added a commit
that referenced
this pull request
Sep 19, 2026
…ents PR #476 removed the comments from the files #475 touched and kept what they said in its commit body. Some of those sentences were not implementation notes but behaviours a user or a later build observes, each already witnessed by a named e2e scenario that no requirement cited. Those become requirements here: 42 across eleven areas, seven of them new (change-detection, configure, group-references, record-write, scoped-builds, tupfile-evaluation, variant-builds) and four extended (command-record, dep-scan, record-read, reset). Every discharge names a test that exists and asserts the sentence, and every sentence forbids an implementation; claims whose only test could not fail on them were cut rather than cited: the edge kind of a group written in the inputs section, a multi-rule depfile (the fixture splits on `&&` into two single-rule scans), the words a derived scan drops, an env-driven branch flip whose command text also changes, the branch a phi-node output is recorded against, the plain-reference arm of demand-driven group parsing, and a variant-build routing sentence that belonged to command-record. A different-model review of the drafts found those plus two upstream citations naming a symbol that does not exist (`variant_check`) and one misplaced assignment (`tf.variant` is set in `parse`, not `parse_tupfile`), all corrected. Nineteen candidates restated a requirement that already exists and were dropped; four of them instead join existing requirements as further discharges. The pass-ordering and arithmetic-width notes stay in #476's body as mechanism. Conformance was read, not guessed: a `tup-conformant` reference names an upstream symbol that was opened, and two behaviours found to diverge are recorded as deliberate deviations for the first time -- putup treats an imported env var that later vanishes as unchanged against the recorded value where tup's env_cb takes NULL as a mismatch, and putup includes a file once per guard context where parser_include_file re-parses on every include. Ref: #476 Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Closes #461.
%iand%Ninow spell the rule's own order-only inputs, as tup does, at all three expansion sites (parse, exec, index), with tup's two refusals:%i is only valid in a command stringoutside a command or display string, and%i used in rule pattern and no order-only input files were specifiedon an empty list.%Niselects a written token and never refuses on emptiness. The order-only list is recorded as a thirdTokenListbeside inputs and outputs (PatternFlags,CommandNode,CommandEntry); the operand record grows two head words andINDEX_VERSIONgoes 25 → 26.Verification
i=[a.c]for%i,%Nirefused as unsupported), green after.%i) fails it.sub/<grp>, bins, globs, duplicates,../z.h, generated headers, a bang macro's own order-only list excluded,%1i/%2i, out-of-range, refusals in outputs and extra outputs, display allowed.make check: 937 test cases, 182383 assertions, tidy, format, spec-check green.make iwyu: no dead includes. spec-check: 153 requirements, 0 gaps.Perf gate (cachegrind, examples/bsp, warm build dir, Ir)
putup parseputup -nDry-run command list identical modulo build path (3137 lines). Callgrind attributes the delta to the third list itself (spelling,
TokenList, node resolution, index words) and to a rule's order-only group patterns now being normalized by both the group pre-resolution loop andexpand_inputs, which the old code skipped for typed groups. Deduplicating the loop is a follow-up candidate, not done here.Review
Design attack (design-attorney workflow) before implementation; break-it review on Opus (semantics, incremental/index, code lenses) with adversarial verification after. Semantics and incremental lenses found nothing this diff introduces; the four surviving code findings (single-call helper inlined, stale DESIGN.md paragraph, blank line, a commit sentence) are fixed in the commit.
Pre-existing siblings the review measured, filed separately: #472 (unknown
{bin}expands to nothing where tup refuses), #473 (dir/<grp>from a nested Tupfile spelled project-relative and resolved to a phantom node, shared with%f), #474 (glob-matched order-only input with a metacharacter in its name gets no edge from the macro edge loop).🤖 Generated with Claude Code
https://claude.ai/code/session_01StgwMENEyfBnEoe4pvAdtQ