Skip to content

#1138 - Make adjacent server deployment mode explicit - #1140

Closed
Hzkun001 wants to merge 2 commits into
NASA-AMMOS:developmentfrom
Hzkun001:fix/1138-adjacent-server-mode
Closed

Hzkun001 wants to merge 2 commits into
NASA-AMMOS:developmentfrom
Hzkun001:fix/1138-adjacent-server-mode

Conversation

@Hzkun001

@Hzkun001 Hzkun001 commented Oct 21, 2026 •

Copy link
Copy Markdown

Purpose

Closes #1138. MMGIS currently starts embedded adjacent processes even when its proxy targets separate Compose services. The proxy and local COG paths also change with Docker detection, while a missing service .env terminates the embedded launcher without useful diagnostics.

This change makes startup, routing, internal STAC requests and local COG paths follow ADJACENT_SERVERS_MODE=embedded|external independently of container runtime detection.

Proposed Changes

  • embedded (default): start enabled Python services and proxy to localhost. external: skip local processes and use the existing service names and ports. Both Compose samples explicitly select external mode.
  • Keep service .env files optional on Unix and Windows; retain inherited configuration and service-specific overrides. STAC/TiPG still require POSTGRES_*, and TiTiler-pgSTAC requires PG*. These are deliberately not inferred from MMGIS's DB_*, because the services normally use the separate mmgis-stac database.
  • Align built-in STAC calls, Tile/Data COG paths and Configure with the mode. Configure's development launcher exports only this setting to CRA; backend PORT/HTTPS settings are not imported into the dev server.
  • Permit the fixed embedded ../../Missions/ prefix in the sample URL policy while rejecting extra traversal and incorrect prefixes.
  • Log child exit code/signal and bounded stderr diagnostics. Redact URL credentials/query strings, labelled secrets and known inherited secret values. Discard a truncated first line so a lost URL/key prefix cannot bypass redaction.

Upgrade note: Existing deployments with separate adjacent containers must set ADJACENT_SERVERS_MODE=external. External services must resolve the existing service hostnames and mount mission files at /Missions. The new default is a deliberate behavior change; it is documented rather than silently retaining a runtime-dependent fallback.

Validation

Automated regression coverage

On Node.js 22.20.0, matching .nvmrc:

npm run test:unit -- --workers=1 --reporter=line

1,471 passed, including 28 adjacent-service tests. New HTTP tests use real Express and http-proxy-middleware over ephemeral loopback ports. They verify routing under ROOT_PATH, user/admin gates, expression validation, and acceptance/rejection by the sample COG policy. Regression failures were observed before the fixes, including the 403 for embedded Mission paths and credential fragments at the stderr buffer boundary.

Actual Docker + TiTiler comparison

Tested original startup/proxy/launcher source from 9a04abc5 and patched source in a minimal Express harness using the real Python service, rather than a mocked upstream. Docker 28.3.3, Linux arm64, Node 22.20.0, Python 3.12.15, TiTiler 2.2.0, TiTiler-pgSTAC 3.1.0, FastAPI 0.140.2, python-dotenv 1.2.2. Local raster: a 32×32 tiled GeoTIFF; patched requests used the actual sample.env URL policy.

Source / deployment Service .env Proxy /titiler/healthz Local raster /titiler/cog/info Local TiTiler process
Original, standalone container Absent 504 — Exits; dotenv reports code 2
Original, standalone container Empty file 504 — Healthy on localhost, unused by proxy
Patched, embedded Absent 200 200, correct 32×32 dimensions Started
Patched, embedded Empty file 200 200, correct 32×32 dimensions Started
Patched, external with separate titiler container Absent 200 200, correct 32×32 dimensions Not started

An additional real failure case removes Python from the embedded process PATH: MMGIS records exit code 127 and the not found stderr diagnostic. Temporary test containers and network are cleaned up.

Configure and other checks

  • Started the actual CRA development server and evaluated its served HTML: mode is external. It retains default port 3000 even with backend PORT=8888 in the root .env. Compilation completes with existing source warnings.
  • Production Configure HTML was converted with the actual html2pug tool and rendered through Pug; the external mode is preserved.
  • Both Compose samples parse and explicitly set external mode; Unix launcher environment/exit-code checks and changed-source syntax/whitespace checks pass.
  • A repeated default-worker unit run previously hit the existing shared-fixture pluginCliE2e.spec.js test disable non-required plugin succeeds (1 failed, 1,414 passed, 48 not run). The complete single-worker run above passes.

Remaining validation

The Docker comparison tests the actual adjacent-service modules in a minimal application harness, not the full production MMGIS image or database-backed application E2E. Podman, Windows batch execution and actual STAC/TiPG/TiTiler-pgSTAC database startup have not been exercised. Full npm test requires DB_USER_TEST and DB_PASS_TEST locally.

Upstream workflow runs currently report action_required with no jobs, so this PR does not claim upstream CI is green. Maintainer action is needed to get those checks running. The separate response URL rewriting issue #753 is outside this change.

@devin-ai-integration devin-ai-integration Bot left a comment •

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Note

Newer findings are available below. Devin Review posted a newer report on this PR, in addition to the findings presented here.

Devin Review found 3 potential issues.

Devin Review

Comment thread adjacent-servers/stac/start-stac.sh
Comment thread configure/public/index.html Outdated
Comment on lines +64 to +71
cmd.on("close", (code, signal) => {
logger(
code === 0 ? "warn" : "error",
`[${s.name}] exited with ${signal ? `signal ${signal}` : `code ${code}`}.${
stderrTail ? ` Stderr: ${stderrTail.trim()}` : ""
}`,
"adjacent-servers"
);

@devin-ai-integration devin-ai-integration Bot Oct 21, 2026 •

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟥 Service database passwords exposed in exit logs

When a child reports POSTGRES_PASS=value or PGPASSWORD=value from a service .env, redactStderr leaves the password intact. process.env lacks service-only passwords, and the label matcher excludes these keys. The exit log persists the database credential.

Devin Review


Was this helpful? React with 👍 or 👎 to provide feedback.

@Hzkun001 Hzkun001 closed this Oct 21, 2026

@devin-ai-integration devin-ai-integration Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Devin Review found 1 new potential issue.

Devin Review

Comment on lines +71 to +77
// A partial first line may have lost the prefix identifying a secret.
let diagnostic = stderrTail;
if (stderrTruncated) {
const newline = stderrTail.indexOf("\n");
diagnostic = newline === -1 ? "" : stderrTail.slice(newline + 1);
}
diagnostic = redactStderr(diagnostic.trim()).slice(-4096);

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟥 Truncated stderr can expose secret fragments

When a credential spans lines, stderrTruncated discards only the first partial line. A remaining unlabeled fragment can survive redactStderr and enter the exit log.

Devin Review


Was this helpful? React with 👍 or 👎 to provide feedback.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

1 participant