Skip to content

Infer dependencies only from fields the runtime renders - #471

Merged
jeremymanning merged 1 commit into
mainfrom
fix/renderable-field-boundary
Aug 4, 2026
Merged

Infer dependencies only from fields the runtime renders#471
jeremymanning merged 1 commit into
mainfrom
fix/renderable-field-boundary

Conversation

@jeremymanning

Copy link
Copy Markdown
Member

Addresses critical finding 1 from review: dependency inference scans fields that never render.

Reproduced verbatim

- id: a
  name: "{{ b.result }}"
- id: b
  name: "{{ a.result }}"
Dependency cycle detected: a -> b -> a

Task names are copied verbatim. Nothing substitutes into them. Two inert strings produced a hard compile error.

Cause — mine, in #466

The canonical graph used a blocklist: scan every step key except id, dependencies and the control-flow keys. I wrote the accompanying comment — "everything else the step carries that may hold a template" — without checking what the runtime actually renders.

Because that graph now drives scheduling and execution levels as well as cycle detection, a spurious edge does more than warn: it serialises independent work, invents cycles, and misreports which dependencies were inferred.

The allowlist, derived from the runtime

core/step_fields.py. Every entry cites the code that renders it:

field evidence
parameters ControlSystem._render_task_templates deep-renders it (control_system.py:363)
action same function, when it is a string (:367)
location kept as location_template, resolved when the output is recorded — which is why real examples write location: "./reports/{{ inputs.topic | slugify }}.md"

Nested steps are recursed into and attributed to the enclosing step, since a child is not scheduled independently of its parent. Control-flow keys are renderable too, but are inferred separately so their edges keep the more precise control_flow origin.

The inert set is named explicitly rather than left implied by absence, so a future reader sees the reasoning instead of inferring it from a gap.

Both directions are pinned

Each has already happened in this repo:

The catalogue guards the first. examples/supported/ — which actually execute — guard the second.

Verification

  • Mutation: reverting to the blocklist fails 7 of 14 new tests, including the CLI-level reproduction (the unit tests call build_dependency_graph directly; the CLI goes through ControlFlowCompiler, and One graph decides what runs before what #466 shipped with a mutation only one of those caught)
  • Blocking suite: 882 passed, 13 skipped
  • Lint: clean
  • Catalogue: all 52 listed examples still validate — no edge dropped

Not in this PR

The surviving warning on the repro says {{ b.result }} in name: "will be resolved at runtime" — precisely what will not happen. The template validator has the same too-broad field notion, and should consume this same declaration. Next, alongside the loop-binding contracts.

🤖 Generated with Claude Code

Two steps that never interact failed to compile:

    - id: a
      name: "{{ b.result }}"
    - id: b
      name: "{{ a.result }}"

    Dependency cycle detected: a -> b -> a

Task names are copied verbatim; nothing substitutes into them. Two
inert strings produced a hard compile error.

The cause was a blocklist introduced with the canonical graph in #466:
build_dependency_graph scanned every key it did not specifically
exclude, so it read name, description, metadata and anything else a
step happened to carry. Since that graph now drives scheduling and
execution levels as well as cycle detection, a spurious edge does not
merely warn -- it serialises independent work, invents cycles, and
misreports which dependencies were inferred.

core.step_fields replaces it with an allowlist, and each entry is there
because the runtime was read rather than guessed:

  parameters  ControlSystem._render_task_templates deep-renders it
  action      the same function renders it when it is a string
  location    kept as location_template and resolved when the output is
              recorded, which is why examples write
              location: "./reports/{{ inputs.topic | slugify }}.md"

Nested steps are recursed into and attributed to the enclosing step, a
child not being scheduled independently of its parent. Control-flow
keys are rendered too, but keep their separate, more precise origin.

Both directions are failure modes that have already happened here, so
both are pinned: scanning too much invents cycles, and scanning too
little re-opens #465, where a step ran before the value it needed
existed. The catalogue guards the first; examples/supported/, which
actually run, guard the second.

Reverting to the blocklist fails 7 of the 14 new tests, including the
CLI-level reproduction.

Blocking suite: 882 passed. Catalogue: all 52 still validate.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
@jeremymanning
jeremymanning merged commit ecac71c into main Aug 4, 2026
11 checks passed
@jeremymanning
jeremymanning deleted the fix/renderable-field-boundary branch August 4, 2026 13:32
jeremymanning added a commit that referenced this pull request Aug 4, 2026
#471 established which parts of a step the runtime renders and wired that into
dependency inference, so two inert strings stopped inventing a cycle. Template
validation was still reading those same fields as if they resolved, and got
both directions wrong:

    - id: a
      name: "{{ b.result }}"          # "will be resolved at runtime"
      description: "{{ nosuch }}"     # a hard error

Nothing substitutes into `name`, so the braces reach the log verbatim and the
warning told the reader the opposite of what happens. And a stray brace in
prose *failed the pipeline* -- a false rejection of a document that runs
correctly, which is the class removed by #465, #469 and #472 elsewhere.

Inert fields now produce a warning that says what actually occurs, and nothing
else. Renderable fields are untouched: an undefined name in `parameters`,
`action` or `location` is still an error.

The same defect exists one level up -- a pipeline's own `name:` was rejected
too -- so `INERT_PIPELINE_FIELDS` sits alongside the step set. It is
deliberately shorter than it first was: `version` is schema-constrained to
`\d+\.\d+\.\d+`, so a template there is a real error and calling the field
inert would describe it wrongly. The run in
`test_a_pipeline_with_templates_in_prose_still_runs` is what caught that, and
is the evidence for every field in both sets: a pipeline carrying an
unresolvable reference in each prose field still executes and still writes the
right contents.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant