Skip to content

Stop validating fields the runtime copies verbatim (#104) - #480

Merged
jeremymanning merged 1 commit into
mainfrom
fix/inert-field-diagnostics
Aug 4, 2026
Merged

Stop validating fields the runtime copies verbatim (#104)#480
jeremymanning merged 1 commit into
mainfrom
fix/inert-field-diagnostics

Conversation

@jeremymanning

Copy link
Copy Markdown
Member

Follows #479. Applies the core/step_fields contract — which #471 established and wired into dependency inference — to template validation, which was still reading those fields as if they resolved.

The defect, in both directions

- id: a
  name: "{{ b.result }}"          # "will be resolved at runtime"
  description: "{{ nosuch }}"     # a hard ERROR

A false promise. Nothing substitutes into name; the braces reach the log verbatim, and the warning told the reader the opposite of what happens.

A false rejection, and the more serious half: a stray brace in prose failed the pipeline. That is the class removed by #465, #469 and #472 elsewhere — validation rejecting a document that runs.

Inert fields now produce a warning saying what actually occurs:

'name' is copied verbatim, so this template is never rendered -- the braces appear literally in the output

Renderable fields are untouched: an undefined name in parameters, action or location is still an error. The warning names the inert field rather than the leaf key, so metadata.note reports metadata instead of sending the reader looking for a rule about a key they invented.

The same defect one level up

A pipeline's own name: was rejected too, so INERT_PIPELINE_FIELDS sits beside the step set.

It is deliberately shorter than my first version. I initially included version, and the end-to-end run failed:

ValidationError: '{{ nosuch_version }}' does not match '^\d+\.\d+\.\d+$'

version is schema-constrained, so a template there is a real error and calling the field "inert prose" would describe it wrongly. id is excluded for the same reason. The set is now name, description, metadata — exactly what a real run tolerates an unresolvable reference in.

That is the point of test_a_pipeline_with_templates_in_prose_still_runs: it executes a pipeline carrying an unresolvable reference in every field both sets claim is inert, and asserts the file still lands with the right contents. The declared sets are what execution proves, not what reading the source suggests — the same discipline as #479, and the reason the version mistake did not ship.

Verification

  • ruff check src/orchestrator --select E9,F63,F7,F82,F821,F823,F601,F811 — clean
  • Full blocking suite — 1013 passed, 13 skipped
  • Catalogue — all 52 listed examples still validate
  • Three mutations, each caught by exactly one test:
    • suppression spreads to every step field (the dangerous direction)
    • inert fields skipped silently, with no warning at all
    • inertness fails to propagate into nested values

Rebased onto main after #479 merged, and all three gates re-run on the rebased branch rather than assuming the earlier run still held.

#471 established which parts of a step the runtime renders and wired that into
dependency inference, so two inert strings stopped inventing a cycle. Template
validation was still reading those same fields as if they resolved, and got
both directions wrong:

    - id: a
      name: "{{ b.result }}"          # "will be resolved at runtime"
      description: "{{ nosuch }}"     # a hard error

Nothing substitutes into `name`, so the braces reach the log verbatim and the
warning told the reader the opposite of what happens. And a stray brace in
prose *failed the pipeline* -- a false rejection of a document that runs
correctly, which is the class removed by #465, #469 and #472 elsewhere.

Inert fields now produce a warning that says what actually occurs, and nothing
else. Renderable fields are untouched: an undefined name in `parameters`,
`action` or `location` is still an error.

The same defect exists one level up -- a pipeline's own `name:` was rejected
too -- so `INERT_PIPELINE_FIELDS` sits alongside the step set. It is
deliberately shorter than it first was: `version` is schema-constrained to
`\d+\.\d+\.\d+`, so a template there is a real error and calling the field
inert would describe it wrongly. The run in
`test_a_pipeline_with_templates_in_prose_still_runs` is what caught that, and
is the evidence for every field in both sets: a pipeline carrying an
unresolvable reference in each prose field still executes and still writes the
right contents.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
@jeremymanning
jeremymanning merged commit 619036f into main Aug 4, 2026
11 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant