Skip to content

fix(deploy): show the full command and the tool's real error on failure - #95

Open
autoidm-jon wants to merge 3 commits into
mainfrom
command-errors
Open

autoidm-jon wants to merge 3 commits into
mainfrom
command-errors

Conversation

@autoidm-jon

Copy link
Copy Markdown
Contributor

Why

Derek tested pdt through pdt-meltano and hit two errors that did not say what went wrong. On Azure, pdt printed:

error: pdt az group create --name pdt-shared failed; fix the problem above and re-run

On Google Cloud, pdt printed:

error: pdt gcloud artifacts repositories describe pdt failed while checking resource ownership

Each message cut the command to its first three or four arguments. The Azure line dropped --location eastus, and the Google Cloud line dropped --location eastus2. In both cases the location was the problem. The Google Cloud message also replaced every error other than "not found" with the label "failed while checking resource ownership". That label also covered a bad argument, a missing permission, a disabled API, and an expired login. Derek could not see which command failed or why.

Each provider had its own copy of the failure message, and each copy cut the command in a different place.

Scope

  • deploy_common.fail_command(command, proc, hint="") is now the one way to report a failed command. It prints the tool's captured output, then error: <full command> failed with exit code N; fix the problem above and re-run, then the hint when pdt has one. The command is the line the user can run again, such as pdt az ... or pdt gcloud ....
  • These paths now call it. They lost their own message: Google Cloud run_quiet, run_stream, read_json_or_none, and both gcloud auth login failures. Azure run_quiet (the quota hint goes through hint), run_stream, delete_unless_locked, and az login. AWS docker login/build/push in build_and_push and aws sso login in relogin (the access key advice goes through hint). The shared run_build for docker build/push on Azure CI.
  • read_json_or_none still answers None for "not found". Any other error prints the real gcloud error and the full command.
  • The az CLI runs with python -W ignore. Before, a shell with Python warnings on (Derek's run under Meltano) showed telemetry utcnow DeprecationWarnings and ResourceWarnings around each az error. Errors are not Python warnings, so az still prints them.
  • uv run for the provider scripts and for pdt aws/az/gcloud passes --quiet. That removes the "Downloading ..." and "Installed N packages" lines. uv still prints an error when it cannot install a package.

Tradeoffs

  • The first run of a provider script now prints nothing while uv installs its packages. With a warm uv cache this takes less than one second. On a new computer the Azure CLI download can take longer, with no progress line.
  • Secret values do not reach any command line. They go through stdin or a temporary file, so fail_command does not redact.

Blast Radius

Error output on every cloud provider. A command that succeeds prints the same as before. A command that fails prints the full command and its exit code. The Google Cloud ownership check stops with the real error, as before, but with a different message. The Windows provider does not change. Its WindowsDeployError already includes the PowerShell error text, and its command is an encoded script that the user cannot run again.

Not changed: az_json and describe_json still answer None for any error, and the caller treats None as "the resource does not exist". They have about 50 callers, and some of them depend on that. For example, az account show answers None when no user is signed in. A later write to the same resource then fails through fail_command with the real error. Changing those probes needs a review of each caller.

Verification

  • tests/test_command_errors.py and the new tests in tests/test_google_cloud_not_found.py came first and failed on master. On master, the read_json_or_none test printed failed while checking resource ownership and no --location. They pass at the head. The test, lint, smoke, and build jobs pass on the head, which is rebased onto Release 0.1.3. ruff check . passes.
  • The real gcloud CLI, with an expired login, ran read_json_or_none("artifacts", "repositories", "describe", "pdt", "--location", "eastus2", "--project", ...). Before and after, the gcloud re-login text printed first. The last line changed:
before: error: pdt gcloud artifacts repositories describe pdt failed while checking resource ownership
after:  error: pdt gcloud artifacts repositories describe pdt --location eastus2 --project pdt-nonexistent-proj-xyz --format=json failed with exit code 1; fix the problem above and re-run
  • The real az CLI (2.89.1, in pdt's uv environment) ran run_quiet("group", "show", "--name", "pdt-nonexistent-xyz", "--query", "location") with PYTHONWARNINGS=default. Before, it printed 4 warnings around the error: 2 DeprecationWarnings from telemetry.py, an unclosed file ResourceWarning, and a subprocess ResourceWarning. After, it printed only the error:
ERROR: (ResourceGroupNotFound) Resource group 'pdt-nonexistent-xyz' could not be found.
Code: ResourceGroupNotFound
Message: Resource group 'pdt-nonexistent-xyz' could not be found.
error: pdt az group show --name pdt-nonexistent-xyz --query location failed with exit code 3; fix the problem above and re-run

With PYTHONDEVMODE=1 in place of PYTHONWARNINGS=default, the new code printed the same 4 lines and no warnings.

  • uv run --quiet --script checks: a script with a dependency that does not exist still prints uv's resolution error and exits 1. A script that installs a package prints no "Installed" line. Its own stdout, stderr, and exit code pass through.
  • Not run: a failing Azure group create. It writes to a subscription. tests/test_command_errors.py covers Derek's --location eastus case with his exact Azure error text.

🤖 Generated with Claude Code


Moved from GitLab autoidm/pdt!161 (https://gitlab.com/autoidm/pdt/-/merge_requests/161), opened by autoidm-jon on 2026-09-30.

@github-actions

github-actions Bot commented Oct 2, 2026

Copy link
Copy Markdown

PR tier: tier::architectural (rules and Claude agree)

  • src/pdt/deploy.py is provider-neutral deploy flow
  • src/pdt/deploy_common.py is shared by every provider
  • src/pdt/deploy_common.py adds fail_command and every provider (deploy_aws.py, deploy_aws_batch.py, deploy_azure.py, deploy_google_cloud.py) now routes failures through it, so the shared error wording and exit-code line change for all four at once
  • --quiet added to the uv dispatch in deploy.py:54 and cli.py:411 removes uv's download/install lines, so a first deploy that installs azure-cli now prints nothing for minutes - removed user-facing output measured against the ease-of-use principle
  • fail_command echoes the full command line via shlex.join and now also prints proc.stdout; secrets currently go by temp file (deploy_azure.py:412) or stdin (--data-file -), so nothing leaks today, but that invariant is newly load-bearing and worth a human confirming

A changed diff is scored again. Job log: https://github.com/AutoIDM/pdt/actions/runs/37006898248

autoidm-jon and others added 3 commits November 3, 2026 14:40
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…error

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@autoidm-jon

autoidm-jon commented Nov 3, 2026 •

Copy link
Copy Markdown
Contributor Author

Goal: after a failed cloud command, pdt prints the complete command and the error from the cloud tool, and hides the install progress lines of uv.

Reviewer: Cursor gpt-5.6-luna-high.

Review of the conflict resolutions of the rebase onto main 5a2e564 (old head f7955e4).

# Severity Disposition Finding Action
1 blocker Fixed (rebase) deploy_google_cloud.py: a failed Cloud Build printed a generic message with no command or exit code. Before the rebase, gcloud builds submit went through run_stream, which this PR changed to report the full command; main replaced that path with cloud_build_image. The generic failure now calls fail_command with the full pdt gcloud builds submit ... line and exit code. The missing-role branch keeps main's message. test_a_failed_cloud_build_fails_once asserts the full line and exit code, and fails without the fix.
2 blocker Fixed in part (rebase) deploy_azure_container_apps.py: a failed ACR Tasks build printed pdt az acr build failed with no arguments or exit code, and the two special branches do not print the raw error. Before the rebase, az acr build went through run_stream; main replaced it with cloud_build. The generic failure now calls fail_command with the full pdt az acr build ... line and exit code. test_a_failed_cloud_build_fails_once_and_shows_the_error asserts this, and fails without the fix. The paused-ACR-Tasks and missing-permission branches keep main's messages: each one names the fix, and main's test requires that the paused-tasks text ("file an Azure support request") does not print.
3 blocker Not changed in this rebase; decision for the PR owner deploy_azure.py az_json returns None for every failed command. az_json has the same body at the PR's base and on main, and the PR never changed it. Its callers read None as "absent". Separating "not found" from other failures widens the PR beyond its original scope.
4 blocker Not changed in this rebase; decision for the PR owner deploy_google_cloud.py describe_json returns None for every failed gcloud command. describe_json has the same body at the PR's base and on main, and the PR never changed it. The goal names read_json_or_none (the "failed while checking resource ownership" message), which the PR fixes. describe_json prints no message and does not stop; its callers read IAM policies and execution lists as optional data.
5 should-fix Not changed; no problem deploy_azure.py az(): warnings.simplefilter("ignore") applies to the whole provider process. The provider script is its own process, and pdt emits and reads no Python warnings in it, so the filter hides nothing pdt needs. It matches the PR's original -W ignore on the separate az process. warnings.catch_warnings() changes the same global filter list and is not thread-safe, and az() runs on worker threads, so a scoped version could restore the wrong filters.

Resolutions:

  • deploy_common.py: fail_command ported to main's console markup; run_build calls it.
  • deploy_aws.py, deploy_aws_batch.py: main kept; main removed the code the PR changed, and its docker builds go through run_build.
  • deploy_azure.py: run_quiet, delete_unless_locked, login, and the ReadError path in main() use fail_command; az() ignores the CLI's Python warnings; run_stream dropped (main removed it).
  • deploy_google_cloud.py: run_quiet, read_json_or_none, and both gcloud auth login failures use fail_command; run_stream dropped.
  • deploy_google_cloud.py cloud_build_image and deploy_azure_container_apps.py cloud_build: generic failures use fail_command (findings 1 and 2).
  • cli.py, deploy.py: uv run --quiet added to main's passthrough and provider command.

Checks that passed at b913f12:

  • uv run --group dev ruff check
  • uv run pytest -q -p no:cacheprovider: 2534 passed, 9 skipped (the skips need PowerShell, IntuneManagement, or gcsfs on this computer)

@github-actions

github-actions Bot commented Nov 4, 2026

Copy link
Copy Markdown

PR tier: tier::architectural (rules and Claude agree)

  • src/pdt/deploy.py is provider-neutral deploy flow
  • src/pdt/deploy_common.py is shared by every provider
  • src/pdt/deploy_common.py adds fail_command, a new shared error contract every provider module now depends on, replacing each provider's own failure wording
  • src/pdt/deploy.py:84 and src/pdt/cli.py:810 add --quiet to the provider-neutral uv run --script dispatch, removing uv's install lines from what every command prints
  • src/pdt/deploy_azure.py sets AZ to python -W ignore -m azure.cli and calls warnings.simplefilter("ignore") inside az(), suppressing warnings for the whole provider process

A changed diff is scored again. Job log: https://github.com/AutoIDM/pdt/actions/runs/41847620266

@autoidm-jon

Copy link
Copy Markdown
Contributor Author

Output changes from 5a2e5641 to b913f12a. A fake gcloud on PATH returns a fixed error, in a sample project with provider: google-cloud, project: pdt-sample, and region: eastus2; the real Azure CLI runs with no Azure sign-in. No command signed in or changed a cloud resource.

A failed command prints every argument and its exit code

Commands run: pdt deploy hello-world --plan, with gcloud artifacts repositories describe failing, and again with gcloud services list failing.

The ownership check no longer replaces the error with "failed while checking resource ownership". The eastus2 location that caused the error is now on the line.

 ERROR: (gcloud.artifacts.repositories.describe) INVALID_ARGUMENT: Location eastus2 is not a valid Artifact Registry location.
-error: pdt gcloud artifacts repositories describe pdt failed while checking resource ownership
+error: pdt gcloud artifacts repositories describe pdt --location eastus2 --project pdt-sample --format=json failed with exit code 1; fix the problem above and re-run

Before, the cut after four arguments left --project with no value.

 ERROR: (gcloud.services.list) PERMISSION_DENIED: Permission denied to list services for project pdt-sample.
-error: pdt gcloud services list --enabled --project failed; fix the problem above and re-run the deploy
+error: pdt gcloud services list --enabled --project pdt-sample '--format=value(config.name)' failed with exit code 1; fix the problem above and re-run

uv no longer prints its download and install lines

Command run: the first pdt deploy hello-world --plan with an empty uv cache.

       no .env file in hello-world or a folder above it, so pdt reads env vars only from this terminal
-Downloading grpcio (12.1MiB)
-Downloading duckdb (16.8MiB)
- Downloaded grpcio
- Downloaded duckdb
-Installed 50 packages in 40ms
 checking: Docker on this computer

The Azure CLI's Python warnings no longer print

Command run: pdt az group show --name pdt-sample with PYTHONWARNINGS=default, after uv installed the packages.

The backoff warning comes from the pdt process, not from the Azure CLI, and prints on both versions.

 <pdt>/.venv/lib/python3.15/site-packages/backoff/_decorator.py:201: DeprecationWarning: 'asyncio.iscoroutinefunction' is deprecated and slated for removal in Python 3.16; use inspect.iscoroutinefunction() instead
   if asyncio.iscoroutinefunction(target):
-<uv-cache>/environments-v2/deploy-azure-8e0783fec722ae1f/lib/python3.12/site-packages/azure/cli/core/telemetry.py:295: DeprecationWarning: datetime.datetime.utcnow() is deprecated and scheduled for removal in a future version. Use timezone-aware objects to represent datetimes in UTC: datetime.datetime.now(datetime.UTC).
-  _session.start_time = datetime.datetime.utcnow()
 ERROR: Please run 'az login' to setup account.

Defect: the quoted command does not run again in Windows Command Prompt

This is not fixed in this PR. fail_command builds the line with shlex.join, which uses POSIX shell quotes. Command Prompt does not remove single quotes, so '--format=value(config.name)' reaches gcloud with the quotes in it. A path with a space gets the same quotes.

Not run

These need a cloud sign-in or a cloud change: the Azure run_quiet, delete_unless_locked, az login, and acr build failures; the Cloud Build failure; the AWS docker and aws sso login failures; and a failed gcloud auth login.

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant