Skip to content

✨ Add parameter injection to harness prompt - #99

Closed
dymurray wants to merge 4 commits into
konveyor:mainfrom
dymurray:harnessImprovements
Closed

✨ Add parameter injection to harness prompt#99
dymurray wants to merge 4 commits into
konveyor:mainfrom
dymurray:harnessImprovements

Conversation

@dymurray

@dymurray dymurray commented Aug 4, 2026

Copy link
Copy Markdown
Contributor

This also adds more detailed goose logging.

One thing this does which is completely hacky but for now I'm going to request we leave it which is to allow a user to set KONVEYOR_PARAM_GCP_PROJECT_ID and have it respected for vertex provider until I figure out how to make it work properly.

Summary by CodeRabbit

  • New Features
    • Migration runs now pass configured parameters into prompts and provider setup.
    • Prompts display supplied parameters for improved transparency.
    • Migration diagnostics report tool calls, token usage, stop reasons, and truncated agent messages.
  • Bug Fixes
    • Runtime failures provide clearer details, including captured service logs and additional error information.
    • Successful prompts with no tool calls are correctly treated as failed migration stages.
    • Vertex AI configuration forwards project and location settings correctly.

@coderabbitai

coderabbitai Bot commented Aug 4, 2026

Copy link
Copy Markdown
Contributor

Review Change Stack

Warning

Review limit reached

@dymurray, you've reached your PR review limit, so we couldn't start this review.

Next review available in: 52 minutes

You've used all free OSS reviews for now. Wait for the free limit to reset to keep reviewing this public repository.

How can I continue?

After more reviews become available, a review can be triggered using the @coderabbitai review command as a PR comment. Alternatively, push new commits to this PR.

To avoid repeated limits, reduce automatic review volume by pausing incremental auto-reviews earlier, using label-based review opt-in, excluding WIP or generated PR titles, or requesting reviews manually when the PR is ready. If your team needs uninterrupted high-volume reviews, an organization admin can enable usage-based reviews.

How do review limits work?

CodeRabbit enforces per-developer PR review limits for each organization. Most developers receive the normal plan review availability.

For paid Pro and Pro+ PR reviews, CodeRabbit uses adaptive limits for sustained high-volume activity. When a developer's recent PR review activity reaches the 95th percentile or higher among CodeRabbit users, additional reviews become available more gradually as earlier reviews age out of the rolling window.

Please refer docs for additional details.

Review details
⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Pro Plus

Run ID: 58503301-8174-4f6e-9064-226b92b52fc4

📥 Commits

Reviewing files that changed from the base of the PR and between 4e723c0 and 0c5c44f.

📒 Files selected for processing (1)
  • changes/unreleased/99-harness-params-and-run-diagnostics.yaml
📝 Walkthrough

Walkthrough

The migration harness propagates runtime parameters into prompts and Goose, captures Goose output, records ACP prompt details, and reports additional stage failure conditions.

Changes

Harness parameter propagation and diagnostics

Layer / File(s) Summary
Runtime parameter propagation
harness/internal/config/config.go, harness/internal/prompt/prompt.go, harness/cmd/migration-harness/main.go
Runtime parameters are loaded from KONVEYOR_PARAM_* variables, rendered in prompts in sorted order, and passed into prompt construction.
Goose output capture and provider setup
harness/internal/goose/lifecycle.go, harness/internal/goose/capturewriter_test.go
Goose output is streamed to stderr and retained in a bounded buffer for failure reporting. Diagnostics, GCP parameter forwarding, and capture behavior are configured and tested.
Prompt result and stage failure reporting
harness/internal/acp/session.go, harness/cmd/migration-harness/main.go
Prompt results include tool-call counts and optional RPC error data. The stage logs prompt details, dumps Goose output on failure, suppresses usage text, and fails completed prompts with zero tool calls.

Estimated code review effort: 3 (Moderate) | ~20 minutes

Sequence Diagram(s)

sequenceDiagram
  participant runStage
  participant SendPrompt
  participant ServeProcess
  runStage->>SendPrompt: submit migration prompt
  SendPrompt-->>runStage: return PromptResult with stop reason and tool-call count
  runStage->>ServeProcess: read captured Goose output when the stage fails
  ServeProcess-->>runStage: return output snapshot
Loading

Possibly related PRs

Suggested reviewers: djzager, savitharaghunathan

🚥 Pre-merge checks | ✅ 5
✅ Passed checks (5 passed)
Check name Status Explanation
Title check ✅ Passed The title clearly describes the primary change: adding parameter injection to the harness prompt.
Description check ✅ Passed The description summarizes parameter injection, detailed Goose logging, and Vertex provider parameter support.
Docstring Coverage ✅ Passed No functions found in the changed files to evaluate docstring coverage. Skipping docstring coverage check.
Linked Issues check ✅ Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check ✅ Passed Check skipped because no linked issues were found for this pull request.
✨ Finishing Touches 💡 1
🛠️ Fix failing CI checks 💡
  • Create stacked PR
  • Commit on current branch
🧪 Generate unit tests (beta)
  • Create PR with unit tests

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 2

🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In `@harness/internal/acp/session.go`:
- Around line 204-212: Update the prompt RPC error branch in the session request
flow to preserve and return the partially populated result instead of nil. Set
result.ToolCalls to turnCount before returning the error, including when
msg.Error contains data, so logPromptResult can emit diagnostics for partial
responses; keep the existing error messages unchanged.

In `@harness/internal/goose/lifecycle.go`:
- Around line 42-60: Update captureWriter and its snapshot flow to retain only a
bounded tail of child-process output, preventing unbounded growth in
captureWriter.Write. Define or reuse an appropriate size limit, discard older
bytes as new data arrives, and ensure the failure dump includes a clear
truncation marker whenever earlier output was removed.
🪄 Autofix

Fix all unresolved CodeRabbit comments on this PR:

  • Push a commit to this branch (recommended)
  • Create a new PR with the fixes

ℹ️ Review info
⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Pro Plus

Run ID: a81c305e-3619-4834-960a-886d899aa4bf

📥 Commits

Reviewing files that changed from the base of the PR and between f0d4d62 and 4520e02.

📒 Files selected for processing (5)
  • harness/cmd/migration-harness/main.go
  • harness/internal/acp/session.go
  • harness/internal/config/config.go
  • harness/internal/goose/lifecycle.go
  • harness/internal/prompt/prompt.go

Comment thread harness/internal/acp/session.go Outdated
Comment thread harness/internal/goose/lifecycle.go
dymurray and others added 3 commits August 4, 2026 22:54
Signed-off-by: Dylan <dymurray@redhat.com>
Co-authored-by: coderabbitai[bot] <136622811+coderabbitai[bot]@users.noreply.github.com>
Signed-off-by: Dylan Murray <dymurray@redhat.com>
Signed-off-by: Dylan <dymurray@redhat.com>
@dymurray
dymurray force-pushed the harnessImprovements branch from a8e9dfd to 4e723c0 Compare August 5, 2026 02:57
@coderabbitai

coderabbitai Bot commented Aug 5, 2026

Copy link
Copy Markdown
Contributor

Note

GitHub couldn't provide a complete incremental comparison for this pull request, so CodeRabbit is performing a full review instead. This review may take a little longer.

…stics)

Signed-off-by: Dylan <dymurray@redhat.com>
@dymurray dymurray closed this Aug 10, 2026
djzager added a commit that referenced this pull request Aug 13, 2026
)

## Summary

Three proposed ADRs and CONTEXT.md updates from a design grilling
session covering parameter delivery, skill authoring boundaries, and
execution controls.

### ADR 0009: Parameter Delivery via params.json

Replaces per-parameter `KONVEYOR_PARAM_*` env vars with a single JSON
file at `/run/konveyor/params.json`. Three-section shape:

- `workflow` — workflow-level params (declared on AgentWorkflow,
supplied on AgentWorkflowRun)
- `agent` — agent-level params (declared on Agent, supplied on
AgentRun), type-coerced
- `execution` — resolved execution controls (maxTurns, maxCost,
maxTokens, mode, sessionType)

All prompt-composing text fields (Agent prompt, AgentRun instructions,
AgentWorkflow guide and stage instructions) are rendered as Go templates
by the controller with the resolved parameter map as the data context.

Prior art: Ansible Playbook Bundles (`_apb_plan_parameters`), Ansible
Runner (`extravars`), Tekton (`$(params.name)`).

### ADR 0010: Skill Content Boundary

Refines ADR 0007's stage/domain skill taxonomy with a clear rule:

> A skill contains knowledge and judgment criteria. It never contains
execution control.

**Allowed:** domain knowledge, approach guidance, quality criteria,
output format, judgment calls.
**Not allowed:** reading env vars, counting iterations, branching on
exit codes, running infrastructure tools, git ops, filesystem discovery.

Immediate changes: remove `MAX_FIX_ITERATIONS` from verify skill, remove
env var reads, remove `git add && git commit` instructions.

### ADR 0011: Execution Controls, Mode, and Session Type on CRDs

First-class CRD fields for execution concerns:

| Category | Values | Purpose |
|----------|--------|---------|
| Execution limits | `maxTurns`, `maxCost`, `maxTokens` | Budget
constraints — whichever limit is hit first triggers wind-down |
| Mode | `auto` \| `approve` | Supervision policy — auto is headless,
approve requires human via ACP tee |
| Session type | `plan` \| `execute` | Runtime work pattern — plan uses
native planning, execute does the work |

CRD hierarchy (most specific wins):
- Standalone: `AgentRun > Agent > system default`
- Workflow: `AgentWorkflowRun > Stage > Agent > system default`

Harness enforces limits by monitoring ACP `usage_update` notifications.
Wind-down prompt at ~85-90% of any limit. Usage recorded on AgentRun
status.

### CONTEXT.md

Updated with new terms: Mode, Session Type, Execution Limits, Skill
Content Boundary. Revised AgentRun and Harness entries to reflect
params.json delivery and execution control semantics.

## Related

- PR #99 (parameter injection to harness prompt — the starting point for
this discussion)
- ADR 0007 (harness as thin runner — refined by ADR 0010)
- ADR 0008 (tee topology — mode and usage_update leverage)
- Ian Bolton's HITL design on issues #55, #56 (mode and session type
prior art)

<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit

* **Documentation**
* Expanded guidance on execution controls, parameter resolution,
precedence, and runtime configuration.
* Documented standardized delivery of workflow, agent, and execution
parameters.
  * Clarified the boundary between skill content and execution control.
  * Defined turn and cost limits, supervision modes, and session types.
* Documented usage monitoring, cancellation, wind-down behavior, and
execution reporting.
* Added an unreleased feature entry covering these agent execution
improvements.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->

Signed-off-by: David Zager <david.j.zager@gmail.com>
dymurray pushed a commit to djzager/agentic-controller that referenced this pull request Aug 25, 2026
…onveyor#108)

## Summary

Three proposed ADRs and CONTEXT.md updates from a design grilling
session covering parameter delivery, skill authoring boundaries, and
execution controls.

### ADR 0009: Parameter Delivery via params.json

Replaces per-parameter `KONVEYOR_PARAM_*` env vars with a single JSON
file at `/run/konveyor/params.json`. Three-section shape:

- `workflow` — workflow-level params (declared on AgentWorkflow,
supplied on AgentWorkflowRun)
- `agent` — agent-level params (declared on Agent, supplied on
AgentRun), type-coerced
- `execution` — resolved execution controls (maxTurns, maxCost,
maxTokens, mode, sessionType)

All prompt-composing text fields (Agent prompt, AgentRun instructions,
AgentWorkflow guide and stage instructions) are rendered as Go templates
by the controller with the resolved parameter map as the data context.

Prior art: Ansible Playbook Bundles (`_apb_plan_parameters`), Ansible
Runner (`extravars`), Tekton (`$(params.name)`).

### ADR 0010: Skill Content Boundary

Refines ADR 0007's stage/domain skill taxonomy with a clear rule:

> A skill contains knowledge and judgment criteria. It never contains
execution control.

**Allowed:** domain knowledge, approach guidance, quality criteria,
output format, judgment calls.
**Not allowed:** reading env vars, counting iterations, branching on
exit codes, running infrastructure tools, git ops, filesystem discovery.

Immediate changes: remove `MAX_FIX_ITERATIONS` from verify skill, remove
env var reads, remove `git add && git commit` instructions.

### ADR 0011: Execution Controls, Mode, and Session Type on CRDs

First-class CRD fields for execution concerns:

| Category | Values | Purpose |
|----------|--------|---------|
| Execution limits | `maxTurns`, `maxCost`, `maxTokens` | Budget
constraints — whichever limit is hit first triggers wind-down |
| Mode | `auto` \| `approve` | Supervision policy — auto is headless,
approve requires human via ACP tee |
| Session type | `plan` \| `execute` | Runtime work pattern — plan uses
native planning, execute does the work |

CRD hierarchy (most specific wins):
- Standalone: `AgentRun > Agent > system default`
- Workflow: `AgentWorkflowRun > Stage > Agent > system default`

Harness enforces limits by monitoring ACP `usage_update` notifications.
Wind-down prompt at ~85-90% of any limit. Usage recorded on AgentRun
status.

### CONTEXT.md

Updated with new terms: Mode, Session Type, Execution Limits, Skill
Content Boundary. Revised AgentRun and Harness entries to reflect
params.json delivery and execution control semantics.

## Related

- PR konveyor#99 (parameter injection to harness prompt — the starting point for
this discussion)
- ADR 0007 (harness as thin runner — refined by ADR 0010)
- ADR 0008 (tee topology — mode and usage_update leverage)
- Ian Bolton's HITL design on issues konveyor#55, konveyor#56 (mode and session type
prior art)

<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit

* **Documentation**
* Expanded guidance on execution controls, parameter resolution,
precedence, and runtime configuration.
* Documented standardized delivery of workflow, agent, and execution
parameters.
  * Clarified the boundary between skill content and execution control.
  * Defined turn and cost limits, supervision modes, and session types.
* Documented usage monitoring, cancellation, wind-down behavior, and
execution reporting.
* Added an unreleased feature entry covering these agent execution
improvements.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->

Signed-off-by: David Zager <david.j.zager@gmail.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant