fix(cli): child exposure, explain routing and image rollback (D-0837, D-0512, D-0445, D-0804)

.github/wiki/Config-Env-Vars.md

@@ -110,6 +110,7 @@ These vars control the top-level identity and behavior of a project.
 | `NSELF_UPDATE_CHECK` | `1` to opt in | unset (off) | No | Process environment only. With `1`, a successful command prints at most one stderr line per 24 h when a newer CLI release exists (cached in `~/.nself/cache/update-check.json`, refreshed in the background from `ping.nself.org/version`). Never in JSON mode, CI or with a non-TTY stderr. See [[Update-Notifications]]. |
 | `NSELF_MCP_TOKEN` | string | unset | No | Bearer token for the `sse` and `http` transports of `nself mcp`. Wins over `~/.nself/mcp-token` (mode `0600`). With neither, a v1.5 loopback start generates the file and prints only its path; a v1.4 loopback start serves open with one warning. A non-loopback `--host` needs this or the file plus `--allowed-host`. Never passed to commands the server runs. Secret names (`config.IsSecretKey`, see Masking below) are masked in output and redacted in the machine audit log. See [[MCP-Server]]. |
 | `NSELF_INVOKE_PATH` | string | unset | No | Internal. Set by the machine invoker (MCP, HTTP) on the `nself` child it runs: the command path it checked. The child refuses with `E421` when its own command lookup resolves a different command, and removes the variable before running. Never set it by hand. See [[MCP-Server]]. |
+| `NSELF_INVOKE_TRANSPORT` | string | unset | No | Internal. The invoker sets `mcp`, `http`, or `http-ndjson` so the child can recheck exposure against its mounted tree. Removed before execution. Never set it by hand. |
 | `NSELF_INVOKE_SECRETS` | JSON | unset | No | Internal. Set by the machine invoker (MCP, HTTP) on the `nself` child it runs: a JSON bundle (a nonce and the values) of the secret-marked arguments and flags, so the secrets never appear on the child argv. The child puts the values back before it parses its arguments, then removes the variable. It is ignored unless `NSELF_INVOKED_BY` is set, and a bad or unmatched bundle fails with `E421`. Never set it by hand. See [[MCP-Server]]. |
 | `IMAGE_PINNING` | `lock` or `legacy` | unset (`legacy` in 1.4.x, `lock` from 1.5.0) | No | How generated compose names images. `legacy` emits the pre-lock references (`redis:7-alpine`) when no version is supplied; `lock` emits `repository:version@sha256:<index digest>` from the image lock. In lock mode, an empty `*_VERSION`, the lock version, or the frozen `legacy_ref` tag selects the locked image. Any other `*_VERSION` is used as given, unpinned. To run the pre-lock tag itself, use `IMAGE_PINNING=legacy`. Any `IMAGE_PINNING` value other than `lock` or `legacy` is an error. See [[Image-Lock]]. |
 

.github/wiki/Configuration-Precedence.md

@@ -31,6 +31,8 @@ nself config explain BASE_DOMAIN --json   # one envelope
 
 For example, the command `init` with the flag `--domain` resolves to `BASE_DOMAIN`.
 
+From a monorepo root, `config explain` follows the detected `backend/` or `.backend/` project. Use `nself config explain BASE_DOMAIN --no-monorepo --json` to explain the current directory instead. With `--json`, any redirect notice goes to stderr so stdout remains one JSON document.
+
 The answer lists the env name, the default, the flags that supply the key, the MCP tool and HTTP route parameter that carry each flag, the `nself.yaml` row, every cascade file that sets the key and the winner. A key that is not a known configuration key, and a flag that supplies no key, exit 1 with `E434`. `nself config env explain VAR` walks the same cascade and prints the same winner.
 
 ## Flags that supply a key

.github/wiki/MCP-Server.md

@@ -19,6 +19,8 @@ A machine request names a command from the **machine registry** and gives typed
 
 The child refuses to run anything but the command the invoker checked: the invoker sets `NSELF_INVOKE_PATH` to the classified path, and the child resolves its own argv with the same lookup it is about to execute and answers `E421` when the two differ (for example after a plugin upgrade gave a leaf new subcommands, or the binary was replaced, while the server kept its registry from start-up). The variable is removed before the command runs, so nothing the command spawns inherits it.
 
+After the path matches, the child rebuilds the registry from its mounted command tree and rechecks exposure for the transport and named flags. A plugin upgrade that changes only exposure attributes, such as `surface: cli-only` or removal of JSON output, is refused with `E421` before the handler runs. The internal `NSELF_INVOKE_TRANSPORT` marker distinguishes HTTP document requests from NDJSON streams and is removed before execution.
+
 Running as a child keeps global state, `os.Exit` and the project operation lock identical to a typed command. There is no shell anywhere: every request value is exactly one argv element.
 
 The **machine registry** is the v1.5 registry of the running binary with installed plugin commands mounted. It is the same in both compat modes ([[Compat-V15]]), so tool names and routes are the v1.5 shape in 1.4.x too. The plugin set is read when the server starts; install a plugin, then restart the server.

.github/wiki/pgvector.md

@@ -76,7 +76,7 @@ nself db image switch --rollback         # undo; from any phase
 nself db image switch --rollback --yes   # undo after a finished switch
 ```
 
-`--rollback` writes the previous env values back (the two keys are left set to an empty value), regenerates compose and the lock, starts postgres on the old volume, resets the read-only setting and starts the stack. After a finished switch it needs `--yes`, because rows written since the switch exist only in the new volume. That volume is kept and named; it is never removed for you.
+`--rollback` restores the previous env values, regenerates compose and the lock, starts postgres on the old volume, resets the read-only setting and starts the stack. Previously absent keys are removed; explicitly empty values are preserved. States recorded by older versions retain their recorded empty assignments. After a finished switch it needs `--yes`, because rows written since the switch exist only in the new volume. That volume is kept and named; it is never removed for you.
 
 A second `nself db image switch --to pgvector` on a project that already runs a pgvector image prints that and changes nothing.
 

cmd/commands/config_explain_monorepo_test.go

@@ -0,0 +1,79 @@
+package commands
+
+import (
+	"os"
+	"path/filepath"
+	"strings"
+	"testing"
+
+	"github.com/nself-org/cli/internal/output"
+)
+
+// TestConfigExplainMonorepoExecution exercises Cobra's full hook order against
+// distinct root/backend values. Raw flags must take effect before any redirect,
+// and the redirect notice must never precede the JSON document on stdout.
+func TestConfigExplainMonorepoExecution(t *testing.T) {
+	for _, tc := range []struct {
+		name     string
+		flags    []string
+		want     string
+		redirect bool
+	}{
+		{"disabled", []string{"--no-monorepo"}, "root.test", false},
+		{"default", nil, "backend.test", true},
+		{"explicit false", []string{"--no-monorepo=false"}, "backend.test", true},
+		{"last value wins", []string{"--no-monorepo", "--no-monorepo=false"}, "backend.test", true},
+	} {
+		t.Run(tc.name, func(t *testing.T) {
+			t.Setenv("HOME", t.TempDir())
+			t.Setenv("OTEL_EXPORTER_OTLP_ENDPOINT", "")
+			t.Setenv("ENV", "dev")
+			t.Setenv("NSELF_LEGACY_ENV_ORDER", "")
+			root := envExplainProject(t, map[string]string{".env": "BASE_DOMAIN=root.test\n"})
+			backend := filepath.Join(root, "backend")
+			if err := os.Mkdir(backend, 0o700); err != nil {
+				t.Fatal(err)
+			}
+			if err := os.WriteFile(filepath.Join(backend, ".env"), []byte("BASE_DOMAIN=backend.test\n"), 0o600); err != nil {
+				t.Fatal(err)
+			}
+			for _, name := range []string{"json", "no-monorepo"} {
+				flag := RootCmd.PersistentFlags().Lookup(name)
+				value, changed := flag.Value.String(), flag.Changed
+				if err := flag.Value.Set("false"); err != nil {
+					t.Fatal(err)
+				}
+				flag.Changed = false
+				t.Cleanup(func() { _ = flag.Value.Set(value); flag.Changed = changed })
+			}
+			oldWriter := pilotWriter
+			pilotWriter = func() output.Writer { return output.Writer{Out: os.Stdout, Err: os.Stderr} }
+			output.ResetState()
+			t.Cleanup(func() { pilotWriter = oldWriter; output.ResetState(); RootCmd.SetArgs(nil) })
+			args := append([]string{"config", "explain", "BASE_DOMAIN", "--json", "--reveal"}, tc.flags...)
+			RootCmd.SetArgs(args)
+			var stdout string
+			var runErr error
+			stderr := captureStderr(t, func() {
+				stdout, runErr = captureStdout(t, RootCmd.Execute)
+			})
+			if runErr != nil {
+				t.Fatal(runErr)
+			}
+			data := singleDoc(t, stdout)["data"].(map[string]any)
+			if got := data["effective"].(map[string]any)["value"]; got != tc.want {
+				t.Errorf("effective value = %v, want %s", got, tc.want)
+			}
+			if got := strings.Contains(stderr, "Detected monorepo layout"); got != tc.redirect {
+				t.Errorf("redirect notice = %v, want %v: %s", got, tc.redirect, stderr)
+			}
+			wantDir := root
+			if tc.redirect {
+				wantDir = backend
+			}
+			if cwd, err := os.Getwd(); err != nil || cwd != wantDir {
+				t.Errorf("cwd = %q (%v), want %q", cwd, err, wantDir)
+			}
+		})
+	}
+}

cmd/commands/config_set_restore.go

@@ -40,6 +40,11 @@ type envLineWrite struct {
 
 // write holds interrupts, records the current bytes, then writes the line.
 func (w *envLineWrite) write(key, value string) error {
+	return w.writeEnvPresence(key, value, true)
+}
+
+// writeEnvPresence records the file before setting or removing one dotenv statement.
+func (w *envLineWrite) writeEnvPresence(key, value string, present bool) error {
 	w.holdSignals()
 	data, err := os.ReadFile(w.path)
 	switch {
@@ -52,6 +57,9 @@ func (w *envLineWrite) write(key, value string) error {
 		w.danglingTarget = danglingLinkTarget(w.path)
 	}
 	w.written = true // set before the write: a partial write is restored too
+	if !present {
+		return w.removeEnvLine(key)
+	}
 	w.updated, err = setEnvFileLine(w.path, key, value)
 	return err
 }

cmd/commands/db_image.go

@@ -83,7 +83,11 @@ func runDBImageSwitch(cmd *cobra.Command, _ []string) error {
 	}
 	ctx, stop := signal.NotifyContext(cmdContext(cmd), os.Interrupt, syscall.SIGTERM, syscall.SIGHUP)
 	defer stop()
-	sw := imageswitch.New(imageSwitchConfig(dir, cfg), imageSwitchDepsFor(cmd, dir, asJSON))
+	switchConfig, err := imageSwitchConfig(dir, cfg)
+	if err != nil {
+		return err
+	}
+	sw := imageswitch.New(switchConfig, imageSwitchDepsFor(cmd, dir, asJSON))
 	var rep *imageswitch.Report
 	switch {
 	case plan:

cmd/commands/db_image_deps.go

@@ -15,6 +15,7 @@ import (
 	"context"
 	"fmt"
 	"io"
+	"os"
 	"path/filepath"
 	"sort"
 	"strconv"
@@ -31,12 +32,22 @@ import (
 )
 
 // imageSwitchConfig builds the engine's Config from the loaded project config.
-func imageSwitchConfig(dir string, cfg *config.Config) imageswitch.Config {
+func imageSwitchConfig(dir string, cfg *config.Config) (imageswitch.Config, error) {
+	previous, err := config.ReadEnvFile(filepath.Join(dir, ".env"))
+	if err != nil && !os.IsNotExist(err) {
+		return imageswitch.Config{}, config.EnvReadError(filepath.Join(dir, ".env"), err)
+	}
+	prev := make(map[string]string)
+	for _, key := range []string{imageswitch.EnvImage, imageswitch.EnvDataVolume} {
+		if value, present := previous[key]; present {
+			prev[key] = value
+		}
+	}
 	return imageswitch.Config{
 		ProjectDir: dir, ProjectName: cfg.ProjectName, Service: "postgres", Container: cfg.ProjectName + "_postgres",
 		User: cfg.Postgres.User, Password: cfg.Postgres.Password, TargetFor: pgvectorImageForMajor,
-		PrevEnv: map[string]string{imageswitch.EnvImage: cfg.Postgres.Image, imageswitch.EnvDataVolume: cfg.Postgres.DataVolume},
-	}
+		PrevEnv: prev,
+	}, nil
 }
 
 // pgvectorImageForMajor is the pgvector image compose would run for a Postgres major.
@@ -138,10 +149,17 @@ type envApplier struct {
 }
 
 func (a *envApplier) request(env map[string]string) reconcile.Request {
+	var remove []string
+	for _, key := range []string{imageswitch.EnvImage, imageswitch.EnvDataVolume} {
+		if _, present := env[key]; !present {
+			remove = append(remove, key)
+		}
+	}
 	return reconcile.Request{
 		ProjectDir: a.dir, Command: reconcile.CmdConfigSet,
 		Trigger:    reconcile.Trigger{Kind: reconcile.TriggerConfig, Subject: imageswitch.EnvDataVolume},
 		EnvOverlay: map[string]map[string]string{".env": env}, Stderr: a.stderr, DiffOut: a.diff,
+		EnvRemove: map[string][]string{".env": remove},
 	}
 }
 
@@ -160,12 +178,18 @@ func (a *envApplier) Plan(ctx context.Context, env map[string]string) (any, erro
 
 // Apply writes the lines and regenerates; on failure the env file is restored.
 func (a *envApplier) Apply(ctx context.Context, env map[string]string) error {
-	envFile := filepath.Join(a.dir, ".env")
-	keys := make([]string, 0, len(env))
-	for k := range env {
-		keys = append(keys, k)
+	defer build.SnapshotEnv()()
+	// config.Load exports the switched values. Removed lines must not inherit
+	// those stale process values when reconcile renders the restored cascade.
+	for _, key := range []string{imageswitch.EnvImage, imageswitch.EnvDataVolume} {
+		if _, present := env[key]; !present {
+			if err := os.Unsetenv(key); err != nil {
+				return err
+			}
+		}
 	}
-	sort.Strings(keys)
+	envFile := filepath.Join(a.dir, ".env")
+	keys := []string{imageswitch.EnvImage, imageswitch.EnvDataVolume}
 	var writes []*envLineWrite
 	defer func() {
 		for _, w := range writes {
@@ -178,7 +202,8 @@ func (a *envApplier) Apply(ctx context.Context, env map[string]string) error {
 			for _, k := range keys {
 				w := &envLineWrite{path: envFile}
 				writes = append(writes, w)
-				if err := w.write(k, env[k]); err != nil {
+				value, present := env[k]
+				if err := w.writeEnvPresence(k, value, present); err != nil {
 					return err
 				}
 			}

cmd/commands/db_image_env_remove.go

@@ -0,0 +1,26 @@
+package commands
+
+// Purpose: remove rollback-only dotenv keys using the planner's statement grammar.
+// Inputs: the captured envLineWrite and key. Outputs: a private env file or error.
+// Constraints: the caller holds interrupts and restores captured bytes on failure.
+
+import (
+	"os"
+
+	"github.com/nself-org/cli/internal/reconcile"
+	"github.com/nself-org/cli/internal/security"
+)
+
+func (w *envLineWrite) removeEnvLine(key string) error {
+	if !w.existed {
+		return nil
+	}
+	data, err := reconcile.EnvFileWithoutLine(w.prev, key)
+	if err != nil {
+		return err
+	}
+	if err := os.WriteFile(w.path, data, 0o600); err != nil {
+		return err
+	}
+	return security.EnforceFilePermissions(w.path, 0o600)
+}

cmd/commands/db_image_presence_test.go

@@ -0,0 +1,88 @@
+package commands
+
+import (
+	"bytes"
+	"context"
+	"io"
+	"os"
+	"path/filepath"
+	"testing"
+
+	"github.com/nself-org/cli/internal/config"
+	"github.com/nself-org/cli/internal/postgres/imageswitch"
+)
+
+func TestImageSwitchConfigPreservesAbsentEnvKeys(t *testing.T) {
+	dir := t.TempDir()
+	if err := os.WriteFile(filepath.Join(dir, ".env"), []byte("PROJECT_NAME=p\n"), 0o600); err != nil {
+		t.Fatal(err)
+	}
+	cfg, err := imageSwitchConfig(dir, &config.Config{Postgres: config.PostgresConfig{Image: "inherited:16", DataVolume: "inherited_data"}})
+	if err != nil {
+		t.Fatal(err)
+	}
+	for _, key := range []string{imageswitch.EnvImage, imageswitch.EnvDataVolume} {
+		if _, present := cfg.PrevEnv[key]; present {
+			t.Errorf("absent %s was captured as an assignment; rollback must remove it", key)
+		}
+	}
+}
+
+func TestImageSwitchEnvApplierRestoresPresence(t *testing.T) {
+	for _, previous := range []string{"", "POSTGRES_IMAGE=\"\"\n", "POSTGRES_IMAGE=postgres:16-alpine\nPOSTGRES_DATA_VOLUME=original_data\n"} {
+		t.Run(previous, func(t *testing.T) {
+			restoreProcessEnv(t)
+			for _, key := range []string{imageswitch.EnvImage, imageswitch.EnvDataVolume} {
+				if err := os.Unsetenv(key); err != nil {
+					t.Fatal(err)
+				}
+			}
+			dir := makeNSelfProject(t, "PROJECT_NAME=p\nENV=dev\nBASE_DOMAIN=example.test\nSSL_MODE=none\nMINIO_ENABLED=false\n"+previous)
+			cfg, err := imageSwitchConfig(dir, &config.Config{})
+			if err != nil {
+				t.Fatal(err)
+			}
+			ap := &envApplier{dir: dir, stderr: io.Discard}
+			if err := ap.Apply(context.Background(), cfg.PrevEnv); err != nil {
+				t.Fatal(err)
+			}
+			baselineEnv, err := os.ReadFile(filepath.Join(dir, ".env"))
+			if err != nil {
+				t.Fatal(err)
+			}
+			baseline, err := os.ReadFile(filepath.Join(dir, "docker-compose.yml"))
+			if err != nil {
+				t.Fatal(err)
+			}
+			if err := ap.Apply(context.Background(), map[string]string{imageswitch.EnvImage: "pgvector/pgvector:pg16", imageswitch.EnvDataVolume: "new_data"}); err != nil {
+				t.Fatal(err)
+			}
+			// A separate rollback invocation first loads the switched config.
+			if _, err := config.Load(dir); err != nil {
+				t.Fatal(err)
+			}
+			if err := ap.Apply(context.Background(), cfg.PrevEnv); err != nil {
+				t.Fatal(err)
+			}
+			restored, err := os.ReadFile(filepath.Join(dir, "docker-compose.yml"))
+			if err != nil || !bytes.Equal(restored, baseline) {
+				t.Fatalf("rollback did not regenerate the original compose: %v", err)
+			}
+			restoredEnv, err := os.ReadFile(filepath.Join(dir, ".env"))
+			if err != nil || !bytes.Equal(restoredEnv, baselineEnv) {
+				t.Fatalf("rollback did not restore the original env bytes: %v", err)
+			}
+			got, err := config.ReadEnvFile(filepath.Join(dir, ".env"))
+			if err != nil {
+				t.Fatal(err)
+			}
+			for _, key := range []string{imageswitch.EnvImage, imageswitch.EnvDataVolume} {
+				value, present := got[key]
+				want, had := cfg.PrevEnv[key]
+				if value != want || present != had {
+					t.Errorf("%s = %q present=%v, want %q present=%v", key, value, present, want, had)
+				}
+			}
+		})
+	}
+}