Skip to content

fix: surface HTTP errors instead of silently skipping schema sync - #344

Merged
matthv merged 9 commits into
mainfrom
fix/prd-878-surface-schema-sync-http-errors
Aug 6, 2026
Merged

fix: surface HTTP errors instead of silently skipping schema sync#344
matthv merged 9 commits into
mainfrom
fix/prd-878-surface-schema-sync-http-errors

Conversation

@matthv

@matthv matthv commented Aug 6, 2026

Copy link
Copy Markdown
Member

Context

PRD-878. A client (core-banking-system, staging) was installing a new Forest Admin environment and its schema never reached Forest, with no visible error in their application logs — the Rails app booted normally. Root cause on the client side: FOREST_ENV_SECRET was copied with the variable name included in the value.

Root cause (gem side)

ForestAdminApiRequester builds its Faraday connection without the raise_error middleware, so a non-2xx HTTP response (e.g. a 404 "unknown secret key") never raises an exception. AgentFactory#do_server_want_schema just parsed the JSON body, found no sendSchema key, and treated it as "nothing to send" → logged Info: Schema was not updated since last run. The invalid secret was never detected, in any environment.

The HTTP-status-to-business-error mapping (handle_response_error, 404 → "invalid envSecret", etc.) already existed but was never reached on this path.

Fix

  1. ForestAdminApiRequester#raise_for_response!(response): checks response.success? and raises the same typed errors handle_response_error already maps, without depending on the missing Faraday middleware. Called from both do_server_want_schema and send_schema_to_server.
  2. ForestAdminErrorSubscriber (found while digging further): it dropped every log in production (return if is_production before any log) for ForestExceptions caught by Rails.error.handle at boot — invisible even on a real error. Logging is now unconditional, mapping the real Rails error severity: to the matching Forest logger level. severity: :error is now explicit on the Rails.error.handle call in engine.rb.
  3. Addressed a review comment: the fallback InternalServerError was dropping the original error message from details — restored it (via cause&.message / response.reason_phrase).
  4. AgentFactory#send_schema now validates env_secret (64-char lowercase hex) and auth_secret (must be a string) before syncing — mirroring the Node agent's OptionsValidator. This catches the exact client mistake (a secret copied with the variable name still glued to the value) instantly, without needing network access.
  5. Both that format check and the actual server sync (post_schema) are handled the same way: raise in production, warn and move on everywhere else. A local secret typo, or a well-formed secret the server rejects (wrong project, revoked, network down), no longer blocks dev boot — logs two clear [ForestAdmin] Warn lines and skips the schema sync instead. The rescue is scoped to that HTTP sync call only (not the whole method), so a real local bug — a broken customization, an unwritable schema_path, a malformed append_schema file — still surfaces immediately in every environment instead of being logged as a generic warning (addressed after review feedback).
  6. The validation only fires for a secret that was actually configured (non-nil), not merely present as a settings key, and it's checked from send_schema rather than setup so schema_only_mode (offline generation, e.g. rake forest_admin:schema:generate) never fails on a secret it doesn't need — both addressed after review feedback (Macroscope high-severity finding on the second one).

Resulting behavior (confirmed intentional, see PRD-878)

  • In production, any problem syncing the schema — malformed secret, well-formed secret rejected by the server, network down — crashes the Rails server boot with a clear message, consistent with how every other Forest setup error is already handled.
  • Outside production, the same problems log two clear Warn lines and let the app boot normally, skipping only the schema sync until it's fixed. A genuine local bug (customization, file I/O) is never swallowed — it always raises, in every environment.
  • An agent that was never configured at all (secret left nil) stays completely silent, in every environment — matching the pre-PR behavior.
  • schema_only_mode (offline generation) never validates secrets at all, since it never talks to the server.

Tests

  • 785/785 forest_admin_agent tests green
  • New spec forest_admin_error_subscriber_spec.rb
  • No regression on forest_admin_rails / forest_admin_datasource_customizer (5 pre-existing forest_admin_rails failures confirmed identical on main, unrelated — missing gems locally)
  • Rebased on latest main
  • Reproduced and validated live on real Rails boots, in every combination: invalid-format secret in prod → crash; well-formed-but-wrong secret in prod → crash; either case outside production → warns and boots normally (app still serves traffic); unconfigured forest_admin_rpc_agent bundled in _examples/demo → no spurious warnings; valid secret → clean boot, no regression

Closes PRD-878

🤖 Generated with Claude Code

Note

Surface HTTP errors during schema sync instead of silently skipping

  • Adds raise_for_response! to ForestAdminApiRequester that maps non-2xx HTTP responses to typed exceptions (NotFoundError, BadGatewayError, ServiceUnavailableError, InternalServerError); do_server_want_schema and send_schema_to_server now call this instead of ignoring failures.
  • Validates env_secret format (64-char lowercase hex) and auth_secret type before attempting schema sync; in production, invalid config raises ValidationError, while non-production logs a warning and skips sync.
  • Fixes @has_env_secret to be false when env_secret is nil (previously a nil secret was treated as present).
  • Updates ForestAdminErrorSubscriber to log in all environments (including production) with severity mapping; previously errors were silently dropped in production.
  • Behavioral Change: schema sync errors that were previously swallowed in production are now re-raised, which may surface startup failures that were previously hidden.

Macroscope summarized 9225c8e.

@linear-code

linear-code Bot commented Aug 6, 2026

Copy link
Copy Markdown

PRD-878

matthv and others added 8 commits August 6, 2026 15:59
Faraday has no raise_error middleware configured, so a non-2xx response
(e.g. 404 on an invalid envSecret) never raised — do_server_want_schema
just parsed the JSON body, found no sendSchema key, and treated it as
"nothing to send". Add ForestAdminApiRequester#raise_for_response! to
check the response status explicitly and raise the same typed errors
handle_response_error already maps, and call it from both
do_server_want_schema and send_schema_to_server.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
ForestAdminErrorSubscriber#report returned early whenever is_production
was true, so any ForestException caught by engine.rb's Rails.error.handle
(e.g. a validation error raised during a customization) was reported to
zero logs in production, not even at debug level. Log unconditionally
and map the real Rails error severity to the matching Forest logger
level instead of hardcoding 'Debug'. Also pass severity: :error
explicitly on the Rails.error.handle call, since it defaulted to
:warning for what are actually blocking setup failures.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
raise_for_status dropped the original Faraday error message from the
InternalServerError details for unmapped statuses. Default the message
to cause&.message for handle_response_error's path, and pass the
response's reason_phrase explicitly from raise_for_response!, since
there is no exception there to read a message from.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Mirrors the Node agent's OptionsValidator: env_secret must be a
64-character lowercase hex string and auth_secret must be a string,
checked once at AgentFactory#setup (before any HTTP call is made).
This catches the exact mistake seen in the wild -- an envSecret copied
with the variable name still glued to the value -- instantly and
without needing network access, instead of only surfacing once the
hashcheck request comes back with a 404.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Blocking the whole Rails boot on a config mistake makes sense in
production (see the previous commit), but is too disruptive in dev:
a typo in a local secret used to boot fine before this PR, and should
still. Warn loudly instead ([ForestAdmin] ... 'Warn' log lines) and
skip the schema sync until it's fixed, without raising, in every
non-production environment.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
@has_env_secret checked whether the :env_secret key was present in the
options hash, which is always true for Dry::Configurable settings even
when the value is nil. Any app bundling forest_admin_rpc_agent as a
Gemfile dependency without configuring it (env_secret/auth_secret left
at their nil default) would trigger the new format validation and
crash boot in production, despite never having set up that agent.
Check for a non-nil value instead, restoring the original silent no-op
for a genuinely unconfigured secret.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
validate_secrets_format! ran during AgentFactory#setup, before callers
get a chance to set schema_only_mode. rake forest_admin:schema:generate
sets it right after setup, so a malformed (or placeholder) env_secret
would raise in production and block offline schema generation even
though that mode never syncs to the server and never makes an HTTP
request. Moved the check into send_schema, which schema-only mode never
calls, and dropped the now-unneeded @secrets_format_invalid ivar in
favor of a plain return value.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
…e server

secrets_format_valid? only guarded against a malformed env_secret/
auth_secret; a syntactically valid but wrong (revoked, wrong project,
copy-pasted from another env) secret still surfaces as a raised error
from the actual HTTP call and crashed dev boot the same way it crashes
production, even though the whole point of the earlier dev/prod split
was to never block dev on a Forest connectivity problem. Wrap the
schema generation/send in send_schema with the same rule: re-raise in
production, warn and move on everywhere else. Also switched both
prod/dev checks to read Facades::Container.cache(:is_production)
instead of @options.to_h[:is_production] directly, for consistency
with the rest of the class and to fix two existing specs that stub the
cache directly without going through a full setup call.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
@matthv
matthv force-pushed the fix/prd-878-surface-schema-sync-http-errors branch from f1b80b9 to be4b0dc Compare August 6, 2026 14:00
Comment thread packages/forest_admin_agent/lib/forest_admin_agent/builder/agent_factory.rb Outdated
The previous rescue wrapped the whole method body, so a local bug (a
broken customization, an unwritable schema_path, a malformed
append_schema file) was silently logged as a generic Forest warning
outside production instead of surfacing to the developer. Move the
rescue to wrap only post_schema (the actual hashcheck/send HTTP call),
so generate_schema_file and the append_schema merge keep raising
unconditionally, exactly as they did before this PR, while a
well-formed-but-rejected secret still degrades gracefully in dev.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>

@arnaud-moncel arnaud-moncel left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🔥

@matthv
matthv merged commit ac7a4a0 into main Aug 6, 2026
52 checks passed
@matthv
matthv deleted the fix/prd-878-surface-schema-sync-http-errors branch August 6, 2026 14:22
forest-bot added a commit that referenced this pull request Aug 6, 2026
## [1.37.1](v1.37.0...v1.37.1) (2026-08-06)

### Bug Fixes

* surface HTTP errors instead of silently skipping schema sync ([#344](#344)) ([ac7a4a0](ac7a4a0))
@forest-bot

Copy link
Copy Markdown
Member

🎉 This PR is included in version 1.37.1 🎉

The release is available on GitHub release

Your semantic-release bot 📦🚀

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants