From aa8ea097c0b5a8a93d9d6aa8a519f6f7844dc5d7 Mon Sep 17 00:00:00 2001 From: Kinflou Date: Mon, 16 Nov 2026 01:50:13 +0800 Subject: [PATCH 01/10] Find and run the language server and the CLI everywhere, Flatpak included path mode looked the bare name up with fs.existsSync, which checks the current directory, so it never found anything (#29). The name is now resolved against PATH (PATHEXT on Windows), and the error lists the directories searched. The CLI resolves the same way. A host mode for comline.server.mode and comline.cli.mode runs the program on the host through flatpak-spawn --host, with stdio piped and RUST_LOG passed as --env. Inside a Flatpak, path mode does this on its own, since the sandbox's PATH is not the host's. flatpak-spawn and the program on the host are checked first, and the message says what to install or which setting to change. Generate and Clean run comline generation and comline generation clean, as cli#153 renamed them (#30). The command ids are unchanged. The CLI commands are registered before the server is looked up, so a missing server no longer takes them away. The resolution is in src/resolve.ts, free of vscode, with unit tests run by yarn test in the Package Test job. --- .github/workflows/ci.yml | 3 + .vscodeignore | 1 + CHANGELOG.md | 27 +++ README.md | 132 ++++++------ package.json | 25 ++- src/extension.ts | 218 +++++++++++++------- src/resolve.test.ts | 385 +++++++++++++++++++++++++++++++++++ src/resolve.ts | 419 +++++++++++++++++++++++++++++++++++++++ 8 files changed, 1070 insertions(+), 140 deletions(-) create mode 100644 src/resolve.test.ts create mode 100644 src/resolve.ts diff --git a/.github/workflows/ci.yml b/.github/workflows/ci.yml index acd02d3..e342613 100644 --- a/.github/workflows/ci.yml +++ b/.github/workflows/ci.yml @@ -116,6 +116,9 @@ jobs: - name: Compile TypeScript run: yarn compile + - name: Unit tests + run: yarn test + - name: Test packaging (without binaries) run: npx vsce package --yarn diff --git a/.vscodeignore b/.vscodeignore index 454d1c6..febce1b 100644 --- a/.vscodeignore +++ b/.vscodeignore @@ -16,3 +16,4 @@ scripts/** test-workspace/** bin/README.md RELEASING.md +out/**/*.test.js diff --git a/CHANGELOG.md b/CHANGELOG.md index 951ac4b..4a1a964 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -10,6 +10,15 @@ All notable changes to the "comline-vscode" extension will be documented in this identifier. - README: running a language server other than the bundled one in Flatpak VSCodium and VS Code. +- `host` mode for `comline.server.mode` and `comline.cli.mode`: runs the + language server or the CLI on the host system. Inside a Flatpak that is + `flatpak-spawn --host` (stdio and `RUST_LOG` pass through); the extension + checks `flatpak-spawn` and the program on the host first, and says what to + install or which setting to change. Outside a Flatpak it is the same as + `path`. `comline.server.customPath` / `comline.cli.customPath` name the + program to run in `host` mode, for one that isn't on the host's `PATH` + (`~/.cargo/bin`). +- Unit tests for the server and CLI resolution (`yarn test`), run in CI. - `comline.toml` is its own language, `comline-toml`, highlighted with a bundled TOML grammar (Taplo's, MIT; see `syntaxes/THIRD-PARTY-NOTICES.md`) and served by the Comline language server: diagnostics for its shape and @@ -18,7 +27,25 @@ All notable changes to the "comline-vscode" extension will be documented in this - `f"..."` strings highlight as interpolated, each `{path}` placeholder apart; `{{` and `}}` as escapes. +### Changed +- Inside a Flatpak, `path` mode runs the program on the host through + `flatpak-spawn --host`, since the sandbox's `PATH` is not the host's. The + CLI's default mode is `path`, so Build/Check/Generate/Clean find a `comline` + installed on the host. The bundled server (the server's default) is + unaffected: it runs in the sandbox. +- Generate runs `comline generation` and Clean runs `comline generation clean`, + as cli#153 renamed them; they need a CLI with those commands. The command + ids (`comline.generate`, `comline.clean`) and titles are unchanged. + ### Fixed +- `comline.server.mode: "path"` never worked: the extension passed the bare + name `comline-lsp` to a file-existence check, which looks in the current + directory, not on `PATH`. The name is now looked up on `PATH` (with + `PATHEXT` on Windows), and the error lists the directories searched. The + CLI's `path` mode and a bare `customPath` are resolved the same way, and a + CLI that isn't found is reported before the task runs. +- Build, Check, Generate and Clean were not registered when the language server + wasn't found; they no longer depend on it. - The bundled language server moves from `b980a6e` to `8590a0a`. The old one linked a comline-core from before `T[max N]` bounds and `char`, so it reported "unexpected token `max`" on `char[max 32]`. The new one also reports diff --git a/README.md b/README.md index 1f95093..44bf564 100644 --- a/README.md +++ b/README.md @@ -40,8 +40,8 @@ opens a menu to run the [comline CLI](https://github.com/ComlineProject/cli) against the current workspace, with a toast reporting success or failure: - **Build** — compile and freeze a new version - **Check** — validate without writing anything -- **Generate** — write generated code for every configured target -- **Clean** — remove generated code +- **Generate** — write generated code for every configured target (`comline generation`) +- **Clean** — remove generated code (`comline generation clean`) Each is also available from the Command Palette as `Comline: Build` / `Comline: Check` / `Comline: Generate` / `Comline: Clean`. Requires the @@ -89,7 +89,7 @@ Not published there yet; use a `.vsix` from releases. ## Configuration -The extension supports three modes for locating the language server binary: +The extension supports four modes for locating the language server binary: ### 1. Bundled (Default) The extension includes a pre-built language server binary. No configuration needed! @@ -118,60 +118,69 @@ Specify a custom path to the language server binary: } ``` +### 4. Host (`host`) +Run the language server on the host system: + +```json +{ + "comline.server.mode": "host" +} +``` + +Inside a Flatpak (below) this goes through `flatpak-spawn --host`; anywhere else +the editor is already on the host, and `host` is the same as `path`. Set +`comline.server.customPath` as well to run a binary that isn't on the host's +`PATH`, such as `~/.cargo/bin/comline-lsp` (written out, `~` is not expanded). + +`path` and `custom` look up the program where the editor runs. `path` searches +the editor's own `PATH` (with `PATHEXT` on Windows) and says which directories it +searched if the server isn't there. An editor started from a launcher or a +dock can have a different `PATH` than your shell. + ### Flatpak (VSCodium, VS Code) -The bundled server needs none of this. Read on only to run a different -`comline-lsp` (a newer build, say) in the Flatpak builds of VSCodium -(`com.vscodium.codium`) or VS Code (`com.visualstudio.code`). The editor runs -inside a sandbox, and `path` mode does not find the server there: Flatpak always -unsets or overrides `PATH` for the app -([`flatpak run`](https://docs.flatpak.org/en/latest/flatpak-command-reference.html#flatpak-run)), -so it is the sandbox's, not your shell's. Use `custom` mode with the binary at a -path the sandbox can read: - -1. **Build a binary that runs in the sandbox.** The sandbox has its own runtime, - not your distro's libraries, so a dynamically linked build may not start. The - bundled Linux servers are static musl builds for that reason; build yours the - same way (`rustup target add x86_64-unknown-linux-musl`, a musl C compiler, - then `cargo build --release --target x86_64-unknown-linux-musl` in a - `language-server` checkout; `aarch64-unknown-linux-musl` on arm64). It must - be for your CPU: the host's, not the editor's. -2. **Put it where the sandbox can read it**, one of: - - **Anywhere under your home directory, if the app already has host or home - access.** Both Flathub manifests ask for `--filesystem=host` - ([VSCodium](https://github.com/flathub/com.vscodium.codium/blob/master/com.vscodium.codium.yaml), - [VS Code](https://github.com/flathub/com.visualstudio.code/blob/master/com.visualstudio.code.yaml)), - so a stock install can read `~/.cargo/bin`, for instance. `flatpak info - --show-permissions ` lists what yours has, overrides included. - - **Grant read access to the directory** if you or Flatseal removed it - (`--nofilesystem=host`), or if it lives somewhere `host` leaves out: - ```bash - flatpak override --user --filesystem=/path/to/dir:ro com.vscodium.codium - flatpak override --user --filesystem=/path/to/dir:ro com.visualstudio.code - ``` - Use the line for your editor. `:ro` makes the directory read-only. Paths - under `/usr`, `/etc`, `/bin` and the like are reserved, and a `--filesystem` - for them has no effect - ([reserved paths](https://docs.flatpak.org/en/latest/sandbox-permissions.html#reserved-paths)). - - **Copy it into the app's own data directory**, which the app can always - read: `mkdir -p ~/.var/app//data/comline && cp comline-lsp - ~/.var/app//data/comline/`. That is `$XDG_DATA_HOME` inside the - sandbox. -3. **Point the extension at it** with an absolute path (the setting is used as - written, so `~` is not expanded): - ```json - { - "comline.server.mode": "custom", - "comline.server.customPath": "/home/you/.var/app/com.vscodium.codium/data/comline/comline-lsp" - } - ``` -4. **Restart.** An override applies when the app starts, so quit the editor - completely and reopen it. The extension reads the server settings when it - activates, so after changing them run `Developer: Reload Window`; - `Comline: Restart Language Server` starts the same binary again. - -If the server isn't found, check the path from inside the sandbox: -`flatpak run --command=ls com.vscodium.codium -l /path/to/comline-lsp`. +The bundled server needs none of this: it is a static build and runs inside the +sandbox. Read on only to run a different `comline-lsp` (a newer build, say), or +the `comline` CLI, in the Flatpak builds of VSCodium (`com.vscodium.codium`) or +VS Code (`com.visualstudio.code`). + +The editor runs in a sandbox with its own `PATH` and its own runtime, so it +can't run programs installed on your system. Install the program on the host as +usual, and the extension runs it there with +[`flatpak-spawn --host`](https://docs.flatpak.org/en/latest/flatpak-command-reference.html#flatpak-spawn): + +- **`path` mode does this by itself inside a Flatpak.** For the CLI it is the + default, so Build/Check/Generate/Clean work with `comline` installed on the + host. For the server, set `"comline.server.mode": "path"` (or `"host"`). +- If the program isn't on the host's `PATH` (the host command runs with your + session's `PATH`, which can lack `~/.cargo/bin`), set `"host"` mode and + `customPath` to its absolute path on the host. The extension checks the host + first and tells you what to install or which setting to change. +- It needs `flatpak-spawn` in the sandbox and the permission to talk to + `org.freedesktop.Flatpak` on D-Bus, which the Flathub manifests of both + editors grant + ([VSCodium](https://github.com/flathub/com.vscodium.codium/blob/master/com.vscodium.codium.yaml), + [VS Code](https://github.com/flathub/com.visualstudio.code/blob/master/com.visualstudio.code.yaml): + `--talk-name=org.freedesktop.Flatpak`). `flatpak info --show-permissions + ` lists what yours has; `flatpak override --user + --talk-name=org.freedesktop.Flatpak ` adds it back. +- The server's stdio is passed through, and `RUST_LOG` (the `comline.server.debug` + setting) is forwarded with `--env`. + +**A custom path** (`custom` mode) is read inside the sandbox. Both Flathub +manifests have `--filesystem=host`, so the binary can be anywhere under your home +directory (or another path `host` covers), but not in the host's `/usr`, `/etc`, +`/bin` or `/lib`: those are [reserved +paths](https://docs.flatpak.org/en/latest/sandbox-permissions.html#reserved-paths), +a `--filesystem` for them has no effect, and the host's `/usr` is at +`/run/host/usr`, not `/usr`. A server installed in `/usr/bin` can only be +reached with `host` mode. A binary the sandbox can read must also run there: it +has the Flatpak runtime's libraries, not your distro's, so a dynamically linked +build may not start (the bundled Linux servers are static musl builds for that +reason). `host` mode has neither limit. + +After changing a setting, run `Developer: Reload Window`; the extension reads the +server settings when it activates. ### Additional Settings @@ -188,7 +197,8 @@ If the server isn't found, check the path from inside the sandbox: ### CLI Binary Location The Build/Check/Generate/Clean commands look for `comline` on `PATH` by -default. To use a non-`PATH` install instead: +default (`comline.cli.mode`: `path`; inside a Flatpak, on the host, see +[Flatpak](#flatpak-vscodium-vs-code)). To use a non-`PATH` install instead: ```json { @@ -197,6 +207,13 @@ default. To use a non-`PATH` install instead: } ``` +`"host"` runs it on the host system, as for the server above, with +`comline.cli.customPath` as the program when set. + +Generate runs `comline generation` and Clean runs `comline generation clean`, +so the CLI has to be a version with those commands (cli#153); older ones had +`generate` and `clean`. + ## Usage 1. Open any `.ids` file @@ -236,8 +253,9 @@ protocol UserService { ### Language server not found - **Default mode**: Ensure the bundled binary is included with the extension -- **PATH mode**: Run `which comline-lsp` (Linux/Mac) or `where comline-lsp` (Windows) to verify installation +- **PATH mode**: Run `which comline-lsp` (Linux/Mac) or `where comline-lsp` (Windows) to verify installation. The error message lists the directories searched; an editor started from a launcher can have a different `PATH` than your shell - **Custom mode**: Check that `comline.server.customPath` points to the correct binary +- **Host mode**: the program has to be on the host (see the message the extension shows) - **Flatpak**: see [Flatpak](#flatpak-vscodium-vs-code) ### No syntax highlighting diff --git a/package.json b/package.json index b07f438..97ac3bf 100644 --- a/package.json +++ b/package.json @@ -203,20 +203,22 @@ "enum": [ "bundled", "path", - "custom" + "custom", + "host" ], "default": "bundled", - "description": "How to locate the language server binary", + "description": "How to locate the language server binary. Reload the window after changing it.", "enumDescriptions": [ - "Use the bundled language server (default)", - "Look for 'comline-lsp' in system PATH", - "Use custom path specified in comline.server.customPath" + "Use the bundled language server (default). It is a static build that also runs inside a Flatpak sandbox.", + "Run 'comline-lsp' from PATH. Inside a Flatpak (VSCodium, VS Code) the sandbox cannot see the host's PATH, so this runs it on the host through 'flatpak-spawn --host'.", + "Run the binary at comline.server.customPath, as written (the path must be readable where the editor runs; inside a Flatpak that excludes the host's /usr).", + "Run the language server on the host system: through 'flatpak-spawn --host' inside a Flatpak, and the same as 'path' anywhere else. Uses comline.server.customPath as the program when set, else 'comline-lsp'." ] }, "comline.server.customPath": { "type": "string", "default": "", - "description": "Custom path to the language server binary (used when mode is 'custom')" + "markdownDescription": "Path to the language server binary. Used as written (`~` is not expanded) when `#comline.server.mode#` is `custom`, and as the program to run on the host when it is `host` (an absolute path on the host, for a binary that isn't on the host's PATH, such as one in `~/.cargo/bin`)." }, "comline.server.debug": { "type": "boolean", @@ -237,19 +239,21 @@ "type": "string", "enum": [ "path", - "custom" + "custom", + "host" ], "default": "path", "description": "How to locate the comline CLI binary used by the Build/Check/Generate/Clean commands", "enumDescriptions": [ - "Look for 'comline' in system PATH (default)", - "Use custom path specified in comline.cli.customPath" + "Run 'comline' from PATH (default). Inside a Flatpak (VSCodium, VS Code) the sandbox cannot see the host's PATH, so this runs it on the host through 'flatpak-spawn --host'.", + "Run the binary at comline.cli.customPath, as written (the path must be readable where the editor runs; inside a Flatpak that excludes the host's /usr).", + "Run the CLI on the host system: through 'flatpak-spawn --host' inside a Flatpak, and the same as 'path' anywhere else. Uses comline.cli.customPath as the program when set, else 'comline'." ] }, "comline.cli.customPath": { "type": "string", "default": "", - "description": "Custom path to the comline CLI binary (used when comline.cli.mode is 'custom')" + "markdownDescription": "Path to the comline CLI binary. Used as written (`~` is not expanded) when `#comline.cli.mode#` is `custom`, and as the program to run on the host when it is `host`." } } } @@ -259,6 +263,7 @@ "compile": "tsc -p ./", "watch": "tsc -watch -p ./", "pretest": "yarn compile", + "test": "node --test out/resolve.test.js", "lint": "eslint src --ext ts", "package": "bash scripts/package.sh", "link-binary": "bash scripts/link-binary.sh", diff --git a/src/extension.ts b/src/extension.ts index aa42236..b724efa 100644 --- a/src/extension.ts +++ b/src/extension.ts @@ -1,5 +1,6 @@ import * as path from 'path'; import * as fs from 'fs'; +import * as os from 'os'; import { execFile } from 'child_process'; import { promisify } from 'util'; import { @@ -43,6 +44,19 @@ import { idpValueKindLabel, IdpFieldSchema, } from './idpSchema'; +import { + Tool, + Plan, + Probe, + TOOLS, + effectiveMode, + planProgram, + launchFor, + hostProbeArgs, + interpretHostProbe, + HostProbe, + hostProbeMessage, +} from './resolve'; const execFileAsync = promisify(execFile); @@ -83,7 +97,15 @@ function ensureTargetsLoaded(): Promise { async function fetchTargets(): Promise { try { - const { stdout } = await execFileAsync(resolveCliCommand(), ['targets'], { timeout: 5000 }); + const plan = await resolveProgram('cli'); + if (plan.kind === 'error') { + throw new Error(plan.message); + } + const launch = launchFor(plan, ['targets'], {}, process.env); + const { stdout } = await execFileAsync(launch.command, launch.args, { + timeout: 5000, + env: launch.env, + }); const targets = new Map(); // `rust#1.85.0 (1.85.0 or later; verified up to 1.99, ...)`: the token, // then what versions its generator handles. @@ -134,6 +156,18 @@ export async function activate(context: ExtensionContext): Promise { { scheme: 'file', language: 'comline-package' }, new LanguagesHoverProvider() ), + // The CLI commands don't depend on the language server: they're + // registered before it is looked up, so a server that isn't found leaves + // them working. + commands.registerCommand('comline.build', () => runCliCommand(['build'])), + commands.registerCommand('comline.check', () => runCliCommand(['check'])), + // `comline generation` and `comline generation clean` (cli#153 renamed + // `generate`/`clean`); the command ids stay `comline.generate`/`.clean` + // so keybindings and tasks that use them keep working. + commands.registerCommand('comline.generate', () => runCliCommand(['generation'])), + commands.registerCommand('comline.clean', () => runCliCommand(['generation', 'clean'])), + commands.registerCommand('comline.selectPackage', () => selectPackageRoot()), + commands.registerCommand('comline.packageStatusBarMenu', () => showPackageStatusBarMenu()), // No client-side formatting provider for `.idp` is registered here — // would be redundant and conflicting, not just unnecessary. Since // `documentSelector` below includes `comline-package` (needed to sync @@ -179,34 +213,39 @@ export async function activate(context: ExtensionContext): Promise { }) ); - const serverExecutable = getServerExecutable(context); - - if (!serverExecutable) { - window.showErrorMessage( - 'Comline Language Server not found. Please configure the server path in settings or ensure it is in your PATH.' - ); - return; - } - - // Check if the executable exists and is accessible - if (!fs.existsSync(serverExecutable)) { - window.showErrorMessage( - `Comline Language Server not found at: ${serverExecutable}. Please check your configuration.` - ); + const plan = await resolveProgram('server', context); + if (plan.kind === 'error') { + window.showErrorMessage(plan.message); return; } const config = workspace.getConfiguration('comline'); const debug = config.get('server.debug', false); + // `comline-lsp` builds its tracing subscriber from `RUST_LOG` (see + // `language-server/src/main.rs`'s `EnvFilter::from_default_env()`). Run on + // the host, it doesn't get the editor's environment, so the variable goes + // through `flatpak-spawn --env=`; `launchFor` does either. + // eslint-disable-next-line @typescript-eslint/naming-convention + const launch = launchFor(plan, [], debug ? { RUST_LOG: 'debug' } : {}, process.env, { + watchBus: true, + }); + const processOptions: { env?: Record; cwd?: string } = {}; + if (launch.env) { + processOptions.env = launch.env; + } + if (plan.kind === 'host') { + // `flatpak-spawn --host` runs in the working directory it is started in, + // which has to exist on the host: the home directory does, in the sandbox + // and out of it. + processOptions.cwd = os.homedir(); + } + const serverOptions: ServerOptions = { - command: serverExecutable, + command: launch.command, + args: launch.args, transport: TransportKind.stdio, - // `comline-lsp` builds its tracing subscriber from `RUST_LOG` (see - // `language-server/src/main.rs`'s `EnvFilter::from_default_env()`) — an - // env var name, not a JS property, hence the naming-convention opt-out. - // eslint-disable-next-line @typescript-eslint/naming-convention - options: debug ? { env: { ...process.env, RUST_LOG: 'debug' } } : undefined, + options: Object.keys(processOptions).length > 0 ? processOptions : undefined, }; const clientOptions: LanguageClientOptions = { @@ -264,12 +303,6 @@ export async function activate(context: ExtensionContext): Promise { await client.start(); window.showInformationMessage('Comline Language Server restarted.'); }), - commands.registerCommand('comline.build', () => runCliCommand('build')), - commands.registerCommand('comline.check', () => runCliCommand('check')), - commands.registerCommand('comline.generate', () => runCliCommand('generate')), - commands.registerCommand('comline.clean', () => runCliCommand('clean')), - commands.registerCommand('comline.selectPackage', () => selectPackageRoot()), - commands.registerCommand('comline.packageStatusBarMenu', () => showPackageStatusBarMenu()) ); try { @@ -420,16 +453,16 @@ async function showPackageQuickPick( // shortcut hint for an entry present here. const COMMAND_KEYBINDING_LABELS: Record = {}; -const CLI_ACTIONS: { subcommand: string; label: string }[] = [ - { subcommand: 'build', label: 'Build' }, - { subcommand: 'check', label: 'Check' }, - { subcommand: 'generate', label: 'Generate' }, - { subcommand: 'clean', label: 'Clean' }, +const CLI_ACTIONS: { subcommand: string[]; label: string }[] = [ + { subcommand: ['build'], label: 'Build' }, + { subcommand: ['check'], label: 'Check' }, + { subcommand: ['generation'], label: 'Generate' }, + { subcommand: ['generation', 'clean'], label: 'Clean' }, ]; interface StatusBarMenuItem extends QuickPickItem { action?: 'switch' | 'run'; - subcommand?: string; + subcommand?: string[]; root?: string; } @@ -528,18 +561,78 @@ async function resolvePackageRoot(): Promise { return picked; } -function resolveCliCommand(): string { +/** What the resolution reads from this machine. */ +function systemProbe(): Probe { + return { + platform: process.platform, + env: process.env, + exists: (file) => fs.existsSync(file), + isExecutable: (file) => { + try { + if (!fs.statSync(file).isFile()) { + return false; + } + if (process.platform !== 'win32') { + fs.accessSync(file, fs.constants.X_OK); + } + return true; + } catch { + return false; + } + }, + }; +} + +/** Where the language server or the CLI is and how to run it, from the + * `comline.server.*` / `comline.cli.*` settings (see `resolve.ts`). A plan + * that runs on the host through `flatpak-spawn` has been checked against the + * host: the program is there, and the editor may talk to it. Never throws; + * what went wrong is the `error` plan's message. */ +async function resolveProgram(tool: Tool, context?: ExtensionContext): Promise { + const info = TOOLS[tool]; const config = workspace.getConfiguration('comline'); - const mode = config.get('cli.mode', 'path'); - const customPath = config.get('cli.customPath', ''); - return mode === 'custom' && customPath ? customPath : 'comline'; + const section = tool === 'server' ? 'server' : 'cli'; + const mode = config.get(`${section}.mode`, info.defaultMode); + const customPath = config.get(`${section}.customPath`, ''); + + const effective = effectiveMode(tool, mode, customPath); + if (effective.notice) { + window.showWarningMessage(effective.notice); + } + const bundled = effective.mode === 'bundled' && context ? getBundledServerPath(context) : null; + const plan = planProgram({ tool, mode, customPath, bundled }, systemProbe()); + if (plan.kind !== 'host') { + return plan; + } + + let result: HostProbe; + try { + const { stdout, stderr } = await execFileAsync(plan.spawn, hostProbeArgs(plan.program), { + timeout: 10000, + }); + result = interpretHostProbe({ exitCode: 0, stdout, stderr }); + } catch (err) { + const e = err as { code?: number | string; killed?: boolean; stdout?: string; stderr?: string; message?: string }; + result = interpretHostProbe({ + exitCode: e.killed ? 'timed out' : e.code, + stdout: e.stdout ?? '', + stderr: e.stderr ?? (e.code === undefined ? String(e.message ?? err) : ''), + }); + } + if (!result.ok) { + return { + kind: 'error', + message: hostProbeMessage(tool, plan.program, result, process.env.FLATPAK_ID), + }; + } + return plan; } /** `rootOverride` runs against a specific package without touching the * persisted active-package selection — used by the status bar menu's * per-package actions. Omit it to resolve the active package normally. */ async function runCliCommand( - subcommand: string, + subcommand: string[], extraArgs: string[] = [], rootOverride?: string ): Promise { @@ -559,16 +652,21 @@ async function runCliCommand( // otherwise — a ProcessExecution-spawned task may not get a PTY, so // `CLICOLOR_FORCE` (which `anstream` explicitly honors, bypassing TTY // detection) guarantees color here the same as typing the command by hand. - const args = ['--path', packageRoot, subcommand, ...extraArgs]; + const plan = await resolveProgram('cli'); + if (plan.kind === 'error') { + window.showErrorMessage(plan.message); + return; + } + + const name = subcommand.join(' '); + // eslint-disable-next-line @typescript-eslint/naming-convention + const launch = launchFor(plan, ['--path', packageRoot, ...subcommand, ...extraArgs], { CLICOLOR_FORCE: '1' }, process.env); const task = new Task( - { type: 'comline', subcommand }, + { type: 'comline', subcommand: name }, TaskScope.Workspace, - `Comline: ${subcommand} (${packageLabel(packageRoot)})`, + `Comline: ${name} (${packageLabel(packageRoot)})`, 'comline', - new ProcessExecution(resolveCliCommand(), args, { - // eslint-disable-next-line @typescript-eslint/naming-convention - env: { ...process.env, CLICOLOR_FORCE: '1' }, - }) + new ProcessExecution(launch.command, launch.args, launch.env ? { env: launch.env } : {}) ); task.presentationOptions = { reveal: TaskRevealKind.Always, @@ -582,10 +680,10 @@ async function runCliCommand( } disposable.dispose(); if (e.exitCode === 0) { - window.showInformationMessage(`Comline: ${subcommand} succeeded.`); + window.showInformationMessage(`Comline: ${name} succeeded.`); } else { window.showErrorMessage( - `Comline: ${subcommand} failed (exit ${e.exitCode}). See the terminal for details.` + `Comline: ${name} failed (exit ${e.exitCode}). See the terminal for details.` ); } }); @@ -691,32 +789,6 @@ class LanguagesHoverProvider { } } -function getServerExecutable(context: ExtensionContext): string | null { - const config = workspace.getConfiguration('comline'); - const mode = config.get('server.mode', 'bundled'); - const customPath = config.get('server.customPath', ''); - - switch (mode) { - case 'bundled': - return getBundledServerPath(context); - - case 'path': - return 'comline-lsp'; // Will be searched in system PATH - - case 'custom': - if (!customPath) { - window.showWarningMessage( - 'Comline: Custom server path is not configured. Fallback to bundled server.' - ); - return getBundledServerPath(context); - } - return customPath; - - default: - return getBundledServerPath(context); - } -} - function getBundledServerPath(context: ExtensionContext): string | null { const platform = process.platform; const arch = process.arch; diff --git a/src/resolve.test.ts b/src/resolve.test.ts new file mode 100644 index 0000000..c11cc24 --- /dev/null +++ b/src/resolve.test.ts @@ -0,0 +1,385 @@ +// Run with `yarn test` (node's built-in runner, on the compiled output). + +import { test } from 'node:test'; +import assert from 'node:assert/strict'; +import { + Probe, + effectiveMode, + hostProbeArgs, + hostProbeMessage, + insideFlatpak, + interpretHostProbe, + launchFor, + pathDirectories, + planProgram, + resolveExecutable, +} from './resolve'; + +/** A machine with these files (all executable) and this environment. */ +function machine( + platform: NodeJS.Platform, + files: string[], + env: NodeJS.ProcessEnv = {} +): Probe { + const set = new Set(files); + return { + platform, + env, + exists: (file) => set.has(file), + isExecutable: (file) => set.has(file), + }; +} + +const linux = (files: string[], env: NodeJS.ProcessEnv = {}): Probe => + machine('linux', files, { PATH: '/usr/local/bin:/usr/bin', ...env }); + +// A Flatpak sandbox: the runtime's PATH, FLATPAK_ID, flatpak-spawn present. +const flatpak = (files: string[] = ['/usr/bin/flatpak-spawn'], env: NodeJS.ProcessEnv = {}): Probe => + machine('linux', files, { PATH: '/app/bin:/usr/bin', FLATPAK_ID: 'com.vscodium.codium', ...env }); + +// -- PATH --------------------------------------------------------------- + +test('a name is found in the first PATH directory that has it', () => { + const probe = linux(['/usr/bin/comline-lsp', '/usr/local/bin/comline-lsp']); + assert.deepEqual(resolveExecutable('comline-lsp', probe), { + found: '/usr/local/bin/comline-lsp', + searched: ['/usr/local/bin', '/usr/bin'], + }); +}); + +test('a name that is nowhere reports the directories searched', () => { + assert.deepEqual(resolveExecutable('comline-lsp', linux([])), { + found: undefined, + searched: ['/usr/local/bin', '/usr/bin'], + }); +}); + +test('the current directory is not on PATH just because a file is there', () => { + // The old check was `fs.existsSync('comline-lsp')`, which reads the cwd. + const probe = linux(['comline-lsp', './comline-lsp']); + assert.equal(resolveExecutable('comline-lsp', probe).found, undefined); +}); + +test('empty PATH entries are dropped; an unset PATH is empty', () => { + assert.deepEqual(pathDirectories({ PATH: '/a::/b:' }, 'linux'), ['/a', '/b']); + assert.deepEqual(pathDirectories({}, 'linux'), []); +}); + +test('a name with a directory in it is a path: checked as written, not searched', () => { + const probe = linux(['/opt/comline/comline-lsp']); + assert.deepEqual(resolveExecutable('/opt/comline/comline-lsp', probe), { + found: '/opt/comline/comline-lsp', + searched: [], + }); + assert.equal(resolveExecutable('/opt/missing/comline-lsp', probe).found, undefined); +}); + +test('Windows: PATHEXT supplies the extension, Path is case-insensitive', () => { + const probe = machine('win32', ['C:\\tools\\comline-lsp.exe'], { + Path: 'C:\\Windows;"C:\\tools"', + PATHEXT: '.com;.exe;.bat', + }); + assert.deepEqual(resolveExecutable('comline-lsp', probe), { + found: 'C:\\tools\\comline-lsp.exe', + searched: ['C:\\Windows', 'C:\\tools'], + }); +}); + +test('Windows: extensions are tried in PATHEXT order, a bare file without one is not a program', () => { + const both = machine('win32', ['C:\\t\\comline.bat', 'C:\\t\\comline.exe'], { + PATH: 'C:\\t', + PATHEXT: '.exe;.bat', + }); + assert.equal(resolveExecutable('comline', both).found, 'C:\\t\\comline.exe'); + const bare = machine('win32', ['C:\\t\\comline'], { PATH: 'C:\\t' }); + assert.equal(resolveExecutable('comline', bare).found, undefined); +}); + +test('Windows: a name that already has a PATHEXT extension is tried as it is', () => { + const probe = machine('win32', ['C:\\t\\comline-lsp.exe'], { PATH: 'C:\\t' }); + assert.equal(resolveExecutable('comline-lsp.exe', probe).found, 'C:\\t\\comline-lsp.exe'); +}); + +test('Windows: PATHEXT defaults when unset', () => { + const probe = machine('win32', ['C:\\t\\comline.CMD'], { PATH: 'C:\\t' }); + assert.equal(resolveExecutable('comline', probe).found, 'C:\\t\\comline.CMD'); +}); + +// -- Flatpak ------------------------------------------------------------ + +test('Flatpak is detected by FLATPAK_ID or by /.flatpak-info, on Linux only', () => { + assert.equal(insideFlatpak(linux([], { FLATPAK_ID: 'com.vscodium.codium' })), true); + assert.equal(insideFlatpak(linux(['/.flatpak-info'])), true); + assert.equal(insideFlatpak(linux([])), false); + assert.equal(insideFlatpak(linux([], { FLATPAK_ID: '' })), false); + assert.equal(insideFlatpak(machine('win32', ['/.flatpak-info'], { FLATPAK_ID: 'x' })), false); + assert.equal(insideFlatpak(machine('darwin', [], { FLATPAK_ID: 'x' })), false); +}); + +// -- The mode ----------------------------------------------------------- + +test('an unknown mode is the tool\'s default; the CLI has no bundled mode', () => { + assert.equal(effectiveMode('server', 'nonsense', '').mode, 'bundled'); + assert.equal(effectiveMode('cli', 'nonsense', '').mode, 'path'); + assert.equal(effectiveMode('cli', 'bundled', '').mode, 'path'); + assert.equal(effectiveMode('server', 'host', '').mode, 'host'); + assert.equal(effectiveMode('cli', 'host', '').mode, 'host'); +}); + +test('custom without a path falls back: bundled for the server (with a notice), path for the CLI', () => { + const server = effectiveMode('server', 'custom', ' '); + assert.equal(server.mode, 'bundled'); + assert.match(server.notice ?? '', /Custom server path is not configured/); + assert.deepEqual(effectiveMode('cli', 'custom', ''), { mode: 'path' }); +}); + +// -- Plans -------------------------------------------------------------- + +const server = (mode: string, customPath = '', bundled: string | null = '/ext/bin/comline-lsp') => ({ + tool: 'server' as const, + mode, + customPath, + bundled, +}); +const cli = (mode: string, customPath = '') => ({ tool: 'cli' as const, mode, customPath }); + +test('bundled runs the bundled binary, and says so when there is none', () => { + assert.deepEqual(planProgram(server('bundled'), linux([])), { + kind: 'direct', + command: '/ext/bin/comline-lsp', + }); + const plan = planProgram(server('bundled', '', null), linux([])); + assert.equal(plan.kind, 'error'); +}); + +test('bundled stays bundled inside a Flatpak: the static binary runs in the sandbox', () => { + assert.deepEqual(planProgram(server('bundled'), flatpak()), { + kind: 'direct', + command: '/ext/bin/comline-lsp', + }); +}); + +test('path finds the server on PATH (#29)', () => { + assert.deepEqual(planProgram(server('path'), linux(['/usr/bin/comline-lsp'])), { + kind: 'direct', + command: '/usr/bin/comline-lsp', + }); +}); + +test('path that finds nothing names the PATH it searched', () => { + const plan = planProgram(server('path'), linux([])); + assert.equal(plan.kind, 'error'); + assert.match((plan as { message: string }).message, /'comline-lsp' was not found on PATH/); + assert.match((plan as { message: string }).message, /\/usr\/local\/bin:\/usr\/bin/); +}); + +test('path for the CLI looks for comline', () => { + assert.deepEqual(planProgram(cli('path'), linux(['/usr/local/bin/comline'])), { + kind: 'direct', + command: '/usr/local/bin/comline', + }); +}); + +test('path inside a Flatpak runs on the host through flatpak-spawn', () => { + assert.deepEqual(planProgram(server('path'), flatpak()), { + kind: 'host', + spawn: '/usr/bin/flatpak-spawn', + program: 'comline-lsp', + }); + assert.deepEqual(planProgram(cli('path'), flatpak()), { + kind: 'host', + spawn: '/usr/bin/flatpak-spawn', + program: 'comline', + }); +}); + +test('path inside a Flatpak without flatpak-spawn says so, and what to set instead', () => { + const plan = planProgram(server('path'), flatpak([])); + assert.equal(plan.kind, 'error'); + const message = (plan as { message: string }).message; + assert.match(message, /flatpak-spawn/); + assert.match(message, /comline\.server\.mode/); +}); + +test('host inside a Flatpak runs the name, or the custom path, on the host', () => { + assert.deepEqual(planProgram(server('host'), flatpak()), { + kind: 'host', + spawn: '/usr/bin/flatpak-spawn', + program: 'comline-lsp', + }); + assert.deepEqual(planProgram(server('host', '/home/me/.cargo/bin/comline-lsp'), flatpak()), { + kind: 'host', + spawn: '/usr/bin/flatpak-spawn', + program: '/home/me/.cargo/bin/comline-lsp', + }); +}); + +test('host outside a Flatpak is this machine: PATH, or the custom path', () => { + assert.deepEqual(planProgram(cli('host'), linux(['/usr/bin/comline'])), { + kind: 'direct', + command: '/usr/bin/comline', + }); + assert.deepEqual(planProgram(cli('host', '/opt/c/comline'), linux(['/opt/c/comline'])), { + kind: 'direct', + command: '/opt/c/comline', + }); + assert.equal(planProgram(cli('host', '/opt/c/comline'), linux([])).kind, 'error'); +}); + +test('custom runs the path as written', () => { + assert.deepEqual(planProgram(server('custom', '/opt/c/comline-lsp'), linux(['/opt/c/comline-lsp'])), { + kind: 'direct', + command: '/opt/c/comline-lsp', + }); +}); + +test('custom that is not there says so; in a Flatpak it points at the reserved paths and host mode', () => { + const outside = planProgram(server('custom', '/opt/c/comline-lsp'), linux([])); + assert.equal(outside.kind, 'error'); + assert.doesNotMatch((outside as { message: string }).message, /host mode|'host'/); + + const inside = planProgram(server('custom', '/usr/bin/comline-lsp'), flatpak()); + assert.equal(inside.kind, 'error'); + const message = (inside as { message: string }).message; + assert.match(message, /\/usr/); + assert.match(message, /comline\.server\.mode to 'host'/); +}); + +test('custom with a bare name is looked up on PATH', () => { + assert.deepEqual(planProgram(cli('custom', 'comline-dev'), linux(['/usr/bin/comline-dev'])), { + kind: 'direct', + command: '/usr/bin/comline-dev', + }); +}); + +test('custom with no path falls back to bundled (server) or path (CLI)', () => { + assert.deepEqual(planProgram(server('custom'), linux([])), { + kind: 'direct', + command: '/ext/bin/comline-lsp', + }); + assert.deepEqual(planProgram(cli('custom'), linux(['/usr/bin/comline'])), { + kind: 'direct', + command: '/usr/bin/comline', + }); +}); + +// -- Command lines ------------------------------------------------------ + +test('a direct launch is the command, with the environment extended only when asked', () => { + const plan = { kind: 'direct' as const, command: '/usr/bin/comline-lsp' }; + assert.deepEqual(launchFor(plan, [], {}, { A: '1' }), { + command: '/usr/bin/comline-lsp', + args: [], + env: undefined, + }); + assert.deepEqual(launchFor(plan, [], { RUST_LOG: 'debug' }, { A: '1', B: undefined }), { + command: '/usr/bin/comline-lsp', + args: [], + env: { A: '1', RUST_LOG: 'debug' }, + }); +}); + +test('a host launch for the server: stdio stays piped, RUST_LOG goes through --env', () => { + const plan = { kind: 'host' as const, spawn: '/usr/bin/flatpak-spawn', program: 'comline-lsp' }; + assert.deepEqual(launchFor(plan, [], { RUST_LOG: 'debug' }, { PATH: '/app/bin' }, { watchBus: true }), { + command: '/usr/bin/flatpak-spawn', + args: ['--host', '--watch-bus', '--env=RUST_LOG=debug', 'comline-lsp'], + env: undefined, + }); + assert.deepEqual(launchFor(plan, [], {}, {}, { watchBus: true }).args, [ + '--host', + '--watch-bus', + 'comline-lsp', + ]); +}); + +test('a host launch for the CLI passes its arguments after the program', () => { + const plan = { kind: 'host' as const, spawn: 'flatpak-spawn', program: '/home/me/.cargo/bin/comline' }; + assert.deepEqual( + launchFor(plan, ['--path', '/work/pkg', 'generation', 'clean'], { CLICOLOR_FORCE: '1' }, {}), + { + command: 'flatpak-spawn', + args: [ + '--host', + '--env=CLICOLOR_FORCE=1', + '/home/me/.cargo/bin/comline', + '--path', + '/work/pkg', + 'generation', + 'clean', + ], + env: undefined, + } + ); +}); + +// -- Checking the host -------------------------------------------------- + +test('the host probe asks the host\'s shell, with the program as a plain argument', () => { + assert.deepEqual(hostProbeArgs('comline-lsp'), [ + '--host', + 'sh', + '-c', + 'command -v "$1"', + 'sh', + 'comline-lsp', + ]); + // Never spliced into the script, so a path with spaces or quotes is inert. + assert.equal(hostProbeArgs('/a b/"c"')[3], 'command -v "$1"'); +}); + +test('the host probe: found, missing, or flatpak-spawn itself failing', () => { + assert.deepEqual(interpretHostProbe({ exitCode: 0, stdout: '/usr/bin/comline-lsp\n', stderr: '' }), { + ok: true, + }); + assert.deepEqual(interpretHostProbe({ exitCode: 1, stdout: '', stderr: '' }), { + ok: false, + reason: 'missing', + }); + assert.deepEqual(interpretHostProbe({ exitCode: 127, stdout: '', stderr: '\n' }), { + ok: false, + reason: 'missing', + }); + assert.deepEqual(interpretHostProbe({ exitCode: 0, stdout: '', stderr: '' }), { + ok: false, + reason: 'missing', + }); + assert.deepEqual( + interpretHostProbe({ + exitCode: 1, + stdout: '', + stderr: 'Portal call failed: org.freedesktop.DBus.Error.ServiceUnknown\n', + }), + { + ok: false, + reason: 'failed', + detail: 'Portal call failed: org.freedesktop.DBus.Error.ServiceUnknown', + } + ); + assert.deepEqual(interpretHostProbe({ exitCode: 'ENOENT', stdout: '', stderr: '' }), { + ok: false, + reason: 'failed', + detail: 'ENOENT', + }); +}); + +test('the host messages say what to install and which setting to change', () => { + const missing = hostProbeMessage('server', 'comline-lsp', { ok: false, reason: 'missing' }); + assert.match(missing, /not found on the host/); + assert.match(missing, /cargo install --git https:\/\/github\.com\/ComlineProject\/language-server/); + assert.match(missing, /comline\.server\.customPath/); + assert.match(missing, /comline\.server\.mode to 'bundled'/); + + const cliMissing = hostProbeMessage('cli', 'comline', { ok: false, reason: 'missing' }); + assert.match(cliMissing, /ComlineProject\/cli/); + assert.doesNotMatch(cliMissing, /bundled/); + + const failed = hostProbeMessage( + 'server', + 'comline-lsp', + { ok: false, reason: 'failed', detail: 'denied' }, + 'com.vscodium.codium' + ); + assert.match(failed, /denied/); + assert.match(failed, /--talk-name=org\.freedesktop\.Flatpak com\.vscodium\.codium/); +}); diff --git a/src/resolve.ts b/src/resolve.ts new file mode 100644 index 0000000..cc51eb9 --- /dev/null +++ b/src/resolve.ts @@ -0,0 +1,419 @@ +// Where the language server and the CLI are, and how to run them. +// +// Pure on purpose: nothing here imports `vscode` or touches the disk except +// through the `Probe` it is handed, so `resolve.test.ts` can run it under +// `node --test` with a fake file system, environment and platform. +// +// The modes (`comline.server.mode`, `comline.cli.mode`): +// +// bundled the server shipped in the extension (server only) +// path the program's name, looked up on PATH (the editor's own PATH: +// inside a Flatpak that is the sandbox's, which cannot see the +// host's programs, so there it runs on the host instead) +// custom the path in `.customPath` +// host the program on the host system, through `flatpak-spawn --host` +// when the editor runs in a Flatpak; outside one, the host is +// this machine and it is `path`/`custom` resolution + +import * as path from 'path'; + +export type Tool = 'server' | 'cli'; +export type Mode = 'bundled' | 'path' | 'custom' | 'host'; + +interface ToolInfo { + /** What the messages call it. */ + label: string; + /** The program's name. */ + name: string; + /** The settings' prefix: `.mode`, `.customPath`. */ + prefix: string; + install: string; + modes: Mode[]; + defaultMode: Mode; +} + +export const TOOLS: Record = { + server: { + label: 'language server', + name: 'comline-lsp', + prefix: 'comline.server', + install: 'cargo install --git https://github.com/ComlineProject/language-server', + modes: ['bundled', 'path', 'custom', 'host'], + defaultMode: 'bundled', + }, + cli: { + label: 'CLI', + name: 'comline', + prefix: 'comline.cli', + install: 'cargo install --git https://github.com/ComlineProject/cli', + modes: ['path', 'custom', 'host'], + defaultMode: 'path', + }, +}; + +export const FLATPAK_SPAWN = 'flatpak-spawn'; + +/** What the resolution reads from the outside world. */ +export interface Probe { + platform: NodeJS.Platform; + env: NodeJS.ProcessEnv; + /** A file that exists and can be run: executable on POSIX, a file on Windows. */ + isExecutable(file: string): boolean; + /** Anything exists at this path. */ + exists(file: string): boolean; +} + +// -- PATH --------------------------------------------------------------- + +function pathApi(platform: NodeJS.Platform): path.PlatformPath { + return platform === 'win32' ? path.win32 : path.posix; +} + +/** A variable by name; Windows environment names are case-insensitive (`Path`). */ +function envVar(env: NodeJS.ProcessEnv, name: string, platform: NodeJS.Platform): string | undefined { + if (platform !== 'win32') { + return env[name]; + } + const key = Object.keys(env).find((k) => k.toUpperCase() === name.toUpperCase()); + return key === undefined ? undefined : env[key]; +} + +/** The directories of PATH, in order. Empty entries are dropped: POSIX reads + * one as the current directory, which is not where a program should be + * found from. */ +export function pathDirectories(env: NodeJS.ProcessEnv, platform: NodeJS.Platform): string[] { + const raw = envVar(env, 'PATH', platform) ?? ''; + const separator = platform === 'win32' ? ';' : ':'; + return raw + .split(separator) + .map((dir) => (platform === 'win32' ? dir.trim().replace(/^"(.*)"$/, '$1') : dir)) + .filter((dir) => dir !== ''); +} + +/** The suffixes that make a Windows file runnable (`PATHEXT`); POSIX has none. */ +export function pathExtensions(env: NodeJS.ProcessEnv, platform: NodeJS.Platform): string[] { + if (platform !== 'win32') { + return ['']; + } + const raw = envVar(env, 'PATHEXT', platform); + const list = raw && raw.trim() !== '' ? raw : '.COM;.EXE;.BAT;.CMD'; + return list + .split(';') + .map((ext) => ext.trim()) + .filter((ext) => ext !== ''); +} + +function hasSeparator(name: string, platform: NodeJS.Platform): boolean { + return platform === 'win32' ? /[\\/]/.test(name) : name.includes('/'); +} + +export interface Found { + /** The program's path, or undefined when it isn't there. */ + found: string | undefined; + /** The directories looked in (empty for a name that is already a path). */ + searched: string[]; +} + +/** A bare name against PATH (with `PATHEXT` on Windows); a name with a + * directory in it is a path and is only checked to exist. */ +export function resolveExecutable(name: string, probe: Probe): Found { + const { platform, env } = probe; + if (hasSeparator(name, platform)) { + return { found: probe.exists(name) ? name : undefined, searched: [] }; + } + const api = pathApi(platform); + const extensions = pathExtensions(env, platform); + const lower = name.toLowerCase(); + const candidates = + platform === 'win32' + ? [ + ...(extensions.some((ext) => lower.endsWith(ext.toLowerCase())) ? [name] : []), + ...extensions.map((ext) => name + ext), + ] + : [name]; + const searched = pathDirectories(env, platform); + for (const dir of searched) { + for (const candidate of candidates) { + const file = api.join(dir, candidate); + if (probe.isExecutable(file)) { + return { found: file, searched }; + } + } + } + return { found: undefined, searched }; +} + +// -- Flatpak ------------------------------------------------------------ + +/** Whether this process runs in a Flatpak sandbox. Flatpak sets `FLATPAK_ID` + * to the app's id and mounts the instance's metadata at `/.flatpak-info` + * (https://docs.flatpak.org/en/latest/flatpak-command-reference.html#flatpak-run, + * https://docs.flatpak.org/en/latest/flatpak-command-reference.html#flatpak-metadata); + * either one says so. */ +export function insideFlatpak(probe: Probe): boolean { + if (probe.platform !== 'linux') { + return false; + } + return (probe.env.FLATPAK_ID ?? '') !== '' || probe.exists('/.flatpak-info'); +} + +// -- The mode ----------------------------------------------------------- + +export interface EffectiveMode { + mode: Mode; + /** Something to tell the user about how the setting was read. */ + notice?: string; +} + +/** The mode a setting asks for: an unknown value is the default; `custom` + * with no path falls back (the bundled server, or `path` for the CLI). */ +export function effectiveMode(tool: Tool, mode: string, customPath: string): EffectiveMode { + const info = TOOLS[tool]; + const asked = (info.modes as string[]).includes(mode) ? (mode as Mode) : info.defaultMode; + if (asked === 'custom' && customPath.trim() === '') { + return tool === 'server' + ? { mode: 'bundled', notice: 'Comline: Custom server path is not configured. Fallback to bundled server.' } + : { mode: 'path' }; + } + return { mode: asked }; +} + +// -- The plan ----------------------------------------------------------- + +export type Plan = + /** Run this file as it is. */ + | { kind: 'direct'; command: string } + /** Run `program` on the host: `spawn` (flatpak-spawn) with `--host`. */ + | { kind: 'host'; spawn: string; program: string } + | { kind: 'error'; message: string }; + +export interface PlanInput { + tool: Tool; + /** `.mode` as read. */ + mode: string; + /** `.customPath` as read. */ + customPath: string; + /** The bundled server's path, null when it isn't in the extension. */ + bundled?: string | null; +} + +const RESERVED = '/usr, /etc, /bin and /lib'; + +function delimiterFor(platform: NodeJS.Platform): string { + return platform === 'win32' ? ';' : ':'; +} + +function notOnPath(tool: Tool, searched: string[], platform: NodeJS.Platform): string { + const { label, name, prefix, install } = TOOLS[tool]; + const where = searched.length > 0 ? searched.join(delimiterFor(platform)) : '(PATH is empty)'; + const alternatives = + tool === 'server' + ? `set ${prefix}.mode to 'bundled', or 'custom' with ${prefix}.customPath` + : `set ${prefix}.mode to 'custom' with ${prefix}.customPath`; + return ( + `Comline: the ${label} '${name}' was not found on PATH. Searched: ${where}. ` + + `Install it (${install}) and make sure the editor sees it: an editor started from ` + + `a launcher has a different PATH than your shell. Or ${alternatives}.` + ); +} + +function notAtCustomPath(tool: Tool, customPath: string, inFlatpak: boolean): string { + const { label, prefix } = TOOLS[tool]; + return ( + `Comline: the ${label} was not found at '${customPath}' (${prefix}.customPath). ` + + 'Check the setting: it is used as written, so `~` is not expanded.' + + (inFlatpak + ? ` The editor runs in a Flatpak sandbox, which cannot see the host's ${RESERVED} ` + + `(a --filesystem permission for them has no effect): a program installed there ` + + `cannot be reached by a path. Set ${prefix}.mode to 'host' to run it on the host.` + : '') + ); +} + +function noFlatpakSpawn(tool: Tool, searched: string[], platform: NodeJS.Platform): string { + const { label, prefix } = TOOLS[tool]; + const alternatives = + tool === 'server' + ? `'bundled', or 'custom' with a path the sandbox can read` + : `'custom' with a path the sandbox can read`; + return ( + `Comline: running the ${label} on the host needs '${FLATPAK_SPAWN}', and it was not found ` + + `in the sandbox (searched: ${searched.join(delimiterFor(platform)) || '(PATH is empty)'}). ` + + `Set ${prefix}.mode to ${alternatives}.` + ); +} + +/** What to run for a tool, given its settings and the world around it. Does + * not run anything: a `host` plan still has to be checked against the host + * (`hostProbeArgs`, `interpretHostProbe`). */ +export function planProgram(input: PlanInput, probe: Probe): Plan { + const info = TOOLS[input.tool]; + const customPath = input.customPath.trim(); + const { mode } = effectiveMode(input.tool, input.mode, customPath); + const inFlatpak = insideFlatpak(probe); + + const onHost = (program: string): Plan => { + if (!inFlatpak) { + // No sandbox in the way: the host is where the editor is. + const { found, searched } = resolveExecutable(program, probe); + if (found) { + return { kind: 'direct', command: found }; + } + return { + kind: 'error', + message: + searched.length > 0 || !hasSeparator(program, probe.platform) + ? notOnPath(input.tool, searched, probe.platform) + : notAtCustomPath(input.tool, program, false), + }; + } + const spawn = resolveExecutable(FLATPAK_SPAWN, probe); + if (!spawn.found) { + return { kind: 'error', message: noFlatpakSpawn(input.tool, spawn.searched, probe.platform) }; + } + return { kind: 'host', spawn: spawn.found, program }; + }; + + switch (mode) { + case 'bundled': + return input.bundled + ? { kind: 'direct', command: input.bundled } + : { + kind: 'error', + message: + `Comline: the bundled ${info.label} is not in this installation. Set ` + + `${info.prefix}.mode to 'path', 'custom' or 'host', or install the extension ` + + `package for your platform.`, + }; + + case 'custom': { + const { found, searched } = resolveExecutable(customPath, probe); + if (found) { + return { kind: 'direct', command: found }; + } + return { + kind: 'error', + message: hasSeparator(customPath, probe.platform) + ? notAtCustomPath(input.tool, customPath, inFlatpak) + : notOnPath(input.tool, searched, probe.platform), + }; + } + + case 'host': + // `customPath`, when set, is the program's path on the host: the host's + // PATH is the session's, not a shell's, so `~/.cargo/bin` is often not + // on it. + return onHost(customPath || info.name); + + case 'path': + default: + // In a Flatpak the sandbox's PATH is not the host's, so a name can only + // mean the host's; elsewhere `onHost` is the PATH lookup. + return onHost(info.name); + } +} + +// -- Running a plan ----------------------------------------------------- + +export interface Launch { + command: string; + args: string[]; + /** The environment for the process, when it isn't the editor's own. */ + env: Record | undefined; +} + +function definedEnv(env: NodeJS.ProcessEnv): Record { + const out: Record = {}; + for (const [key, value] of Object.entries(env)) { + if (value !== undefined) { + out[key] = value; + } + } + return out; +} + +/** The command line for a plan. `extraEnv` is what the program needs on top + * of the editor's environment (`RUST_LOG`, `CLICOLOR_FORCE`): set on the + * process when it runs here, and as `--env=` for the host, which does not get + * the sandbox's environment. `watchBus` makes the host process exit when + * `flatpak-spawn` does, so a long-running server doesn't outlive the editor. */ +export function launchFor( + plan: Exclude, + args: string[], + extraEnv: Record, + baseEnv: NodeJS.ProcessEnv, + options: { watchBus?: boolean } = {} +): Launch { + if (plan.kind === 'direct') { + return { + command: plan.command, + args, + env: Object.keys(extraEnv).length > 0 ? { ...definedEnv(baseEnv), ...extraEnv } : undefined, + }; + } + return { + command: plan.spawn, + args: [ + '--host', + ...(options.watchBus ? ['--watch-bus'] : []), + ...Object.entries(extraEnv).map(([key, value]) => `--env=${key}=${value}`), + plan.program, + ...args, + ], + env: undefined, + }; +} + +// -- Checking the host -------------------------------------------------- + +/** `flatpak-spawn` arguments that ask the host's shell where `program` is: + * `command -v` prints the path of an executable, by name or by path, and + * nothing when there is none. */ +export function hostProbeArgs(program: string): string[] { + return ['--host', 'sh', '-c', 'command -v "$1"', 'sh', program]; +} + +export type HostProbe = + | { ok: true } + | { ok: false; reason: 'missing' } + | { ok: false; reason: 'failed'; detail: string }; + +/** What `hostProbeArgs`' run says. A nonzero exit with nothing on stderr is + * `command -v` finding nothing; anything on stderr is `flatpak-spawn` itself + * failing (no permission to talk to `org.freedesktop.Flatpak`, no session + * helper), and is passed on. */ +export function interpretHostProbe(result: { + exitCode: number | string | null | undefined; + stdout: string; + stderr: string; +}): HostProbe { + const stderr = result.stderr.trim(); + if (result.exitCode === 0) { + return result.stdout.trim() !== '' ? { ok: true } : { ok: false, reason: 'missing' }; + } + if (typeof result.exitCode === 'number' && stderr === '') { + return { ok: false, reason: 'missing' }; + } + return { ok: false, reason: 'failed', detail: stderr || String(result.exitCode ?? 'no exit status') }; +} + +export function hostProbeMessage(tool: Tool, program: string, probe: HostProbe, appId?: string): string { + const { label, prefix, install } = TOOLS[tool]; + if (probe.ok) { + return ''; + } + if (probe.reason === 'missing') { + return ( + `Comline: '${program}' was not found on the host. Install the ${label} there (${install}) ` + + `so it is on the host's PATH, or set ${prefix}.customPath to its absolute path on the host ` + + `(the host command runs with your session's PATH, which may not include ~/.cargo/bin).` + + (tool === 'server' ? ` Or set ${prefix}.mode to 'bundled'.` : '') + ); + } + return ( + `Comline: '${FLATPAK_SPAWN} --host' failed: ${probe.detail}. Running on the host needs the ` + + `editor to be allowed to talk to org.freedesktop.Flatpak (Flatpak's own VSCodium and VS Code ` + + `have it; \`flatpak override --user --talk-name=org.freedesktop.Flatpak ${appId ?? ''}\` ` + + `grants it).` + ); +} From edda3ee81d0c641255e9a3229abc7e98a00cc864 Mon Sep 17 00:00:00 2001 From: Kinflou Date: Mon, 16 Nov 2026 02:06:26 +0800 Subject: [PATCH 02/10] ci: run the extension's resolution in Flathub's VSCodium sandbox A workflow that installs com.vscodium.codium from Flathub on an Ubuntu VM and runs out/resolve.js inside it, with a real flatpak-spawn --host: the bundled server in the sandbox, host mode (by name, by a path in the host's /usr, by a path in the home directory, RUST_LOG, --watch-bus), the host probe, path and custom resolution, and the CLI's generation and generation clean on disk. The system probe and the host check move into resolve.ts so the extension and the test run the same code. --- .github/workflows/flatpak.yml | 313 +++++++++++++++++++++++++++ scripts/flatpak-e2e.cjs | 395 ++++++++++++++++++++++++++++++++++ src/extension.ts | 50 +---- src/resolve.ts | 62 ++++++ 4 files changed, 771 insertions(+), 49 deletions(-) create mode 100644 .github/workflows/flatpak.yml create mode 100644 scripts/flatpak-e2e.cjs diff --git a/.github/workflows/flatpak.yml b/.github/workflows/flatpak.yml new file mode 100644 index 0000000..d012052 --- /dev/null +++ b/.github/workflows/flatpak.yml @@ -0,0 +1,313 @@ +name: Flatpak + +# Proves what the README promises about Flatpak, so nobody has to check it by +# hand: the extension's real program resolution (out/resolve.js, the module the +# extension loads) runs inside Flathub's real VSCodium sandbox, and starts the +# language server and the CLI on the host through a real `flatpak-spawn --host`. +# scripts/flatpak-e2e.cjs lists the scenarios; each one fails the job. +# +# 1. bundled the static comline-lsp runs in the sandbox, answers initialize +# 2. host comline-lsp runs on the host (found by name; by a path in the +# host-only /usr; by a path in the home directory): initialize +# works, RUST_LOG=debug reaches it, a clean stop ends it, and +# killing flatpak-spawn ends it too (--watch-bus) +# 3. host probe present, missing, and flatpak-spawn itself failing +# 4. path inside the sandbox resolves to the host +# 5. custom a path under the host's /usr is an error that says `host` +# 6. CLI `comline generation` and `generation clean`, through the +# launch the extension builds, change the files on disk +# +# What it needs, and why it is built this way: +# +# - A VM. Flatpak's sandbox (bubblewrap, user namespaces) does not run inside +# our CI container images. `ubuntu-24.04` is the VM the other workflows use. +# - `flatpak` from apt, at run time. EXCEPTION to the org's rule against +# installing packages while a job runs: the sandbox under test IS Flatpak, +# the runner image does not have it, and there is no pinned image that does. +# So the packages are pinned to exact versions (below) and the install is +# retried on network failure. Ubuntu keeps only its current security update +# of a package, so a version here goes stale when Ubuntu ships the next one: +# the weekly run turns that into a failed run that says which pin to move. +# - `com.vscodium.codium` from Flathub, user installation: the real app, with +# the permissions Flathub gives it, not a stand-in. It follows Flathub's +# current build, on purpose: a weekly run is how a change to its permissions +# (--talk-name=org.freedesktop.Flatpak, --filesystem=host) is noticed. The +# installed commit is logged, and the install is cached by that commit. +# - `dbus-run-session`: `flatpak-spawn --host` talks to the session helper +# over the session bus; a runner has none. +# - comline-lsp: the one the latest GitHub release ships (the linux-x64 +# package's bundled binary), not built again here. Which release is logged. +# - comline (CLI): built here from a pinned commit. It has no release yet. + +on: + pull_request: + paths: + - "src/resolve.ts" + - "src/extension.ts" + - "package.json" + - ".github/workflows/flatpak.yml" + - "scripts/flatpak-e2e.cjs" + push: + branches: [main] + paths: + - "src/resolve.ts" + - "src/extension.ts" + - "package.json" + - ".github/workflows/flatpak.yml" + - "scripts/flatpak-e2e.cjs" + schedule: + # Mondays: Flathub and Ubuntu move without a change here. + - cron: "23 5 * * 1" + workflow_dispatch: + +permissions: + contents: read + +# A newer push to the same pull request supersedes this run; a branch's runs +# queue instead. +concurrency: + group: ${{ github.workflow }}-${{ github.event.pull_request.number || github.ref }} + cancel-in-progress: ${{ github.event_name == 'pull_request' }} + +defaults: + run: + shell: bash + +env: + # ComlineProject/cli, main when this was written. Bump with the CLI's + # command set (it has to have `generation`, cli#153). + CLI_REV: e79092a3cfec9f2b3d02a45189f51e6a90fb8071 + # Ubuntu 24.04 (noble), noble-updates/-security. + FLATPAK_VERSION: 1.14.6-1ubuntu0.1 + BUBBLEWRAP_VERSION: 0.9.0-1ubuntu0.3 + XDG_DBUS_PROXY_VERSION: 0.1.5-1ubuntu0.2 + APP_ID: com.vscodium.codium + +jobs: + cli: + name: Build comline (CLI) + runs-on: ubuntu-24.04 + timeout-minutes: 60 + # The tools image: Rust 1.99.1, the native musl target and musl-gcc. A + # static binary runs on the runner whatever its libc. + container: ghcr.io/comlineproject/ci-tools@sha256:b9301f9caf3e47f0b839de85efda178f698a55daf7fc387b463adeec93ecd433 + steps: + - name: Checkout the CLI at the pinned rev + uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1 + with: + repository: ComlineProject/cli + ref: ${{ env.CLI_REV }} + persist-credentials: false + + - name: musl C compiler + run: | + # comline-core compiles its tree-sitter grammar from C at build time. + echo "CC_x86_64_unknown_linux_musl=musl-gcc" >> "$GITHUB_ENV" + + - uses: Swatinem/rust-cache@d1527c8f67ef9af357290475d4af4b0fab8f12e0 # v2.9.3 + with: + key: x86_64-unknown-linux-musl + + # Rust only: the TypeScript generator and wasmtime are not under test. + - name: Build + run: > + cargo build --release --locked + --target x86_64-unknown-linux-musl + --no-default-features --features gen-rust + + - name: Static, and it runs + run: | + bin=target/x86_64-unknown-linux-musl/release/comline + if readelf -lW "$bin" | grep -q INTERP; then + echo "::error::comline is dynamically linked" + exit 1 + fi + "$bin" targets + + - uses: actions/upload-artifact@cf430e030ddbb5b0abf93d22962f4752f3646cd9 # v7.0.2 + with: + name: comline-cli + path: target/x86_64-unknown-linux-musl/release/comline + if-no-files-found: error + + sandbox: + name: VSCodium sandbox (Flathub) + needs: cli + runs-on: ubuntu-24.04 + timeout-minutes: 45 + steps: + - name: Harden the runner + uses: step-security/harden-runner@1d88384fd173af6ecb012dfd0264a306b711241a # v2.22.4 + with: + egress-policy: audit + + - name: Checkout code + uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1 + with: + persist-credentials: false + + - name: Setup Node.js + uses: actions/setup-node@949feb2413d6458794dcd2491c4babbbce0c15c1 # v7.1.0 + with: + node-version: 20.20.2 + cache: yarn + + - name: Install dependencies + run: yarn install --frozen-lockfile + + # out/resolve.js: what the test imports. + - name: Compile TypeScript + run: yarn compile + + # -- the programs the extension will run ------------------------------- + + - name: The CLI (built above) + uses: actions/download-artifact@9000827ccba6bdab643e8b6fd33ac0654aef8333 # v8.0.2 + with: + name: comline-cli + path: ${{ runner.temp }}/cli + + - name: The language server (the latest release's bundled one) + env: + GH_TOKEN: ${{ github.token }} + run: | + tag=$(gh release list --repo "$GITHUB_REPOSITORY" --limit 1 --json tagName --jq '.[0].tagName') + echo "language server from release $tag" + gh release download "$tag" --repo "$GITHUB_REPOSITORY" \ + --pattern '*-linux-x64.vsix' --dir "$RUNNER_TEMP/release" + mkdir -p "$RUNNER_TEMP/bundled" + unzip -p "$RUNNER_TEMP"/release/*-linux-x64.vsix extension/bin/linux-x64/comline-lsp \ + > "$RUNNER_TEMP/bundled/comline-lsp" + chmod +x "$RUNNER_TEMP/bundled/comline-lsp" + sha256sum "$RUNNER_TEMP/bundled/comline-lsp" + # The bundled server is static, which is why it runs in a sandbox. + if readelf -lW "$RUNNER_TEMP/bundled/comline-lsp" | grep -q INTERP; then + echo "::error::the release's comline-lsp is dynamically linked" + exit 1 + fi + + # -- Flatpak ------------------------------------------------------------- + + # EXCEPTION to "no packages installed while a job runs": see the header. + # Exact versions, retried on a network failure. + - name: Install Flatpak (apt, pinned) + run: | + retry() { + for attempt in 1 2 3 4; do + "$@" && return 0 + echo "attempt $attempt failed: $*" >&2 + sleep $((attempt * 10)) + done + return 1 + } + retry sudo apt-get update -q + retry sudo apt-get install -y -q --no-install-recommends \ + "flatpak=$FLATPAK_VERSION" \ + "bubblewrap=$BUBBLEWRAP_VERSION" \ + "xdg-dbus-proxy=$XDG_DBUS_PROXY_VERSION" + flatpak --version + command -v dbus-run-session + + - name: Flathub, and which build of the app + id: app + run: | + flatpak remote-add --user --if-not-exists flathub https://dl.flathub.org/repo/flathub.flatpakrepo + commit=$(flatpak remote-info --user --show-commit flathub "$APP_ID") + echo "Flathub's current $APP_ID commit: $commit" + echo "commit=$commit" >> "$GITHUB_OUTPUT" + + # Keyed on the app's commit: most runs find it and download nothing. A new + # Flathub build misses, falls back to the latest older cache (the + # runtimes are shared), and updates it. + - name: Flatpak cache + id: cache + uses: actions/cache@55cc8345863c7cc4c66a329aec7e433d2d1c52a9 # v6.1.0 + with: + path: ~/.local/share/flatpak + key: flatpak-${{ env.FLATPAK_VERSION }}-${{ env.APP_ID }}-${{ steps.app.outputs.commit }} + restore-keys: | + flatpak-${{ env.FLATPAK_VERSION }}-${{ env.APP_ID }}- + + - name: Install the app + run: | + flatpak remote-add --user --if-not-exists flathub https://dl.flathub.org/repo/flathub.flatpakrepo + echo "cache hit: ${{ steps.cache.outputs.cache-hit }}" + if [ "${{ steps.cache.outputs.cache-hit }}" = true ]; then + echo "restored from the cache: nothing to download" + else + flatpak install --user --noninteractive --assumeyes flathub "$APP_ID" + # A restored older build is brought up to date. + flatpak update --user --noninteractive --assumeyes "$APP_ID" + fi + flatpak info --user --show-commit "$APP_ID" + flatpak info --user --show-permissions "$APP_ID" + du -sh ~/.local/share/flatpak + + # The documented facts the extension relies on, as the installed app has + # them: if Flathub drops either permission this is where it shows. + - name: The app has the permissions host mode needs + run: | + permissions=$(flatpak info --user --show-permissions "$APP_ID") + grep -E '^filesystems=.*\bhost\b' <<<"$permissions" + grep -E '^org\.freedesktop\.Flatpak=talk' <<<"$permissions" + + # -- the session --------------------------------------------------------- + + - name: Can this runner run a Flatpak sandbox at all? + run: | + sysctl kernel.apparmor_restrict_unprivileged_userns kernel.unprivileged_userns_clone 2>&1 || true + # flatpak wants a runtime dir, which a runner has no login session to make. + sudo mkdir -p "/run/user/$(id -u)" + sudo chown "$(id -u):$(id -g)" "/run/user/$(id -u)" + chmod 700 "/run/user/$(id -u)" + XDG_RUNTIME_DIR="/run/user/$(id -u)" flatpak run --command=sh "$APP_ID" -c 'echo "inside the sandbox: $FLATPAK_ID"; cat /.flatpak-info | head -5' + + - name: Put the programs where the scenarios expect them + run: | + # The host-only /usr: a file the sandbox cannot see at that path. + sudo install -m 755 "$RUNNER_TEMP/bundled/comline-lsp" /usr/local/bin/comline-lsp + # A path under the home directory that is on no PATH. + mkdir -p "$HOME/e2e/hostonly" + install -m 755 "$RUNNER_TEMP/bundled/comline-lsp" "$HOME/e2e/hostonly/comline-lsp" + # The CLI: on the host session's PATH, as ~/.local/bin is for a user. + mkdir -p "$HOME/.local/bin" + install -m 755 "$RUNNER_TEMP/cli/comline" "$HOME/.local/bin/comline" + # A package to generate code from, made by the CLI itself. + (cd "$HOME/e2e" && "$HOME/.local/bin/comline" new tinypkg && ls -R tinypkg) + # What the test reads. + APP_REPO="$GITHUB_WORKSPACE" BUNDLED="$RUNNER_TEMP/bundled/comline-lsp" node -e ' + const home = process.env.HOME; + require("fs").writeFileSync(home + "/e2e/config.json", JSON.stringify({ + repo: process.env.APP_REPO, + bundledLsp: process.env.BUNDLED, + hostLspUsr: "/usr/local/bin/comline-lsp", + hostLspHome: home + "/e2e/hostonly/comline-lsp", + package: home + "/e2e/tinypkg", + appId: process.env.APP_ID, + }, null, 2));' + cat "$HOME/e2e/config.json" + + - name: Find the app's Electron binary + id: electron + run: | + export XDG_RUNTIME_DIR="/run/user/$(id -u)" + flatpak run --command=sh "$APP_ID" -c 'ls -l /app/bin /app/share/codium | head -40' + flatpak run --command=sh "$APP_ID" -c 'test -x /app/share/codium/codium' + echo "bin=/app/share/codium/codium" >> "$GITHUB_OUTPUT" + + # -- the scenarios ------------------------------------------------------- + + - name: Scenarios, inside the sandbox + env: + ELECTRON: ${{ steps.electron.outputs.bin }} + run: | + export XDG_RUNTIME_DIR="/run/user/$(id -u)" + # The host session's PATH, which `flatpak-spawn --host` programs get: + # it is the environment the session bus is started in. + export PATH="$HOME/.local/bin:$PATH" + # Electron as plain Node (no window, no display), in the real sandbox. + dbus-run-session -- flatpak run \ + --command="$ELECTRON" \ + --env=ELECTRON_RUN_AS_NODE=1 \ + "$APP_ID" \ + "$GITHUB_WORKSPACE/scripts/flatpak-e2e.cjs" "$HOME/e2e/config.json" diff --git a/scripts/flatpak-e2e.cjs b/scripts/flatpak-e2e.cjs new file mode 100644 index 0000000..c41ce92 --- /dev/null +++ b/scripts/flatpak-e2e.cjs @@ -0,0 +1,395 @@ +'use strict'; +// The extension's program resolution, run inside Flathub's real VSCodium +// sandbox, against a real `flatpak-spawn --host`. +// +// flatpak run --command=/app/share/codium/codium \ +// --env=ELECTRON_RUN_AS_NODE=1 com.vscodium.codium \ +// scripts/flatpak-e2e.cjs +// +// Run by .github/workflows/flatpak.yml (see there for how the sandbox, the +// session bus and the host's programs are set up). Not a unit test: it needs a +// Flatpak sandbox with `--talk-name=org.freedesktop.Flatpak`, which is what it +// proves the extension works in. The unit tests (`yarn test`) cover the same +// logic with a fake machine; this covers the machine. +// +// It imports the compiled `out/resolve.js`, the very module the extension +// loads, and uses `systemProbe()` (the real file system and environment), so +// it sees the sandbox exactly as the extension does. Nothing here re-implements +// what the extension decides; each scenario asks the extension's code what to +// run, runs that, and checks what happened, on the host as well as in here. +// +// Config (JSON): repo, bundledLsp, hostLspUsr, hostLspHome, package, appId. + +const fs = require('node:fs'); +const os = require('node:os'); +const path = require('node:path'); +const assert = require('node:assert/strict'); +const { spawn, execFile } = require('node:child_process'); +const { promisify } = require('node:util'); + +const execFileAsync = promisify(execFile); +const config = JSON.parse(fs.readFileSync(process.argv[2], 'utf8')); +const r = require(path.join(config.repo, 'out', 'resolve.js')); +const probe = r.systemProbe(); + +const sleep = (ms) => new Promise((resolve) => setTimeout(resolve, ms)); + +function withTimeout(promise, ms, what) { + let timer; + const timeout = new Promise((_, reject) => { + timer = setTimeout(() => reject(new Error(`no ${what} within ${ms / 1000}s`)), ms); + }); + return Promise.race([promise, timeout]).finally(() => clearTimeout(timer)); +} + +// -- The host, through the same door the extension uses ----------------- + +const spawnPath = r.resolveExecutable(r.FLATPAK_SPAWN, probe).found; + +/** A shell script on the host, with positional arguments. */ +async function onHost(script, ...args) { + assert.ok(spawnPath, 'flatpak-spawn is not in the sandbox'); + const { stdout } = await execFileAsync(spawnPath, ['--host', 'sh', '-c', script, 'sh', ...args], { + timeout: 20000, + }); + return stdout; +} + +async function hostServerPids() { + const out = await onHost('pgrep -x comline-lsp || true'); + return out.split('\n').filter((line) => line.trim() !== ''); +} + +async function waitForNoHostServer(what) { + const deadline = Date.now() + 15000; + for (;;) { + const pids = await hostServerPids(); + if (pids.length === 0) { + return; + } + if (Date.now() > deadline) { + throw new Error(`${what}: host comline-lsp still running (pid ${pids.join(', ')}) after 15s`); + } + await sleep(250); + } +} + +// -- A language client, just enough of one ------------------------------ + +function startServer(launch, options = {}) { + const child = spawn(launch.command, launch.args, { + stdio: ['pipe', 'pipe', 'pipe'], + env: launch.env, + ...options, + }); + let stderr = ''; + child.stderr.on('data', (chunk) => { + stderr += chunk; + }); + const exited = new Promise((resolve) => child.on('exit', (code, signal) => resolve({ code, signal }))); + const spawnFailed = new Promise((_, reject) => child.on('error', reject)); + spawnFailed.catch(() => {}); + + let buf = Buffer.alloc(0); + const waiters = []; + child.stdout.on('data', (chunk) => { + buf = Buffer.concat([buf, chunk]); + for (;;) { + const sep = buf.indexOf('\r\n\r\n'); + if (sep < 0) { + return; + } + const len = Number(/Content-Length: (\d+)/i.exec(buf.subarray(0, sep).toString())?.[1]); + if (buf.length < sep + 4 + len) { + return; + } + const msg = JSON.parse(buf.subarray(sep + 4, sep + 4 + len).toString()); + buf = buf.subarray(sep + 4 + len); + for (const w of [...waiters]) { + if (w.match(msg)) { + waiters.splice(waiters.indexOf(w), 1); + w.resolve(msg); + } + } + } + }); + child.stdin.on('error', () => {}); + + const send = (msg) => { + const body = Buffer.from(JSON.stringify({ jsonrpc: '2.0', ...msg })); + child.stdin.write(`Content-Length: ${body.length}\r\n\r\n`); + child.stdin.write(body); + }; + const waitFor = (what, match) => + withTimeout( + Promise.race([ + new Promise((resolve) => waiters.push({ match, resolve })), + spawnFailed, + exited.then((e) => { + throw new Error(`exited early (${e.signal ?? `code ${e.code}`}); stderr: ${stderr.trim()}`); + }), + ]), + 20000, + what + ); + + return { + child, + exited, + stderr: () => stderr, + /** `initialize` as an editor sends it; resolves with the result. */ + async initialize() { + const response = waitFor('initialize response', (m) => m.id === 1); + send({ + id: 1, + method: 'initialize', + params: { processId: process.pid, rootUri: null, capabilities: {} }, + }); + const { result, error } = await response; + assert.ok(!error, `initialize failed: ${JSON.stringify(error)}`); + assert.ok(result && result.capabilities, 'initialize returned no capabilities'); + send({ method: 'initialized', params: {} }); + return result; + }, + /** `shutdown`, `exit`, and the pipe closed, as an editor stops a server. */ + async stop() { + const response = waitFor('shutdown response', (m) => m.id === 2); + send({ id: 2, method: 'shutdown', params: null }); + await response; + send({ method: 'exit', params: null }); + child.stdin.end(); + return withTimeout(exited, 20000, 'exit after `exit`'); + }, + }; +} + +/** A server as the extension would start it: its plan, checked, launched. */ +async function startAsExtension(settings, { debug = false } = {}) { + const planned = r.planProgram({ tool: 'server', bundled: null, customPath: '', ...settings }, probe); + const plan = await r.verifyPlan('server', planned); + assert.notEqual(plan.kind, 'error', plan.message); + const launch = r.launchFor(plan, [], debug ? { RUST_LOG: 'debug' } : {}, process.env, { + watchBus: true, + }); + // The extension runs a host server in the home directory (extension.ts). + const options = plan.kind === 'host' ? { cwd: os.homedir() } : {}; + return { plan, launch, server: startServer(launch, options) }; +} + +/** What the host process is: its binary and whether it got RUST_LOG. */ +async function describeHostServer() { + const pids = await hostServerPids(); + assert.equal(pids.length, 1, `expected one host comline-lsp, found ${pids.length}`); + const exe = (await onHost('readlink -f /proc/$1/exe', pids[0])).trim(); + const environ = await onHost('tr "\\0" "\\n" < /proc/$1/environ', pids[0]); + const rustLog = environ.split('\n').find((line) => line.startsWith('RUST_LOG=')); + return { pid: pids[0], exe, rustLog }; +} + +function walk(dir) { + const files = []; + for (const entry of fs.readdirSync(dir, { withFileTypes: true })) { + const full = path.join(dir, entry.name); + if (entry.isDirectory()) { + files.push(...walk(full)); + } else { + files.push(full); + } + } + return files; +} + +// -- The scenarios ------------------------------------------------------ + +const scenarios = []; +const scenario = (name, fn) => scenarios.push({ name, fn }); + +scenario('this is the Flathub VSCodium sandbox, and the host is reachable', async () => { + assert.equal(r.insideFlatpak(probe), true, 'insideFlatpak() is false in the sandbox'); + assert.equal(process.env.FLATPAK_ID, config.appId); + assert.ok(fs.existsSync('/.flatpak-info'), '/.flatpak-info is missing'); + assert.ok(spawnPath, 'flatpak-spawn is not on the sandbox PATH'); + // The host is another machine's worth of file system: its /usr/local is + // not the sandbox's. + assert.equal(fs.existsSync(config.hostLspUsr), false, `${config.hostLspUsr} is visible in the sandbox`); + await onHost('test -x "$1"', config.hostLspUsr); +}); + +scenario('1. bundled: the static comline-lsp runs in the sandbox and answers initialize', async () => { + const { plan, server } = await startAsExtension({ mode: 'bundled', bundled: config.bundledLsp }); + assert.deepEqual(plan, { kind: 'direct', command: config.bundledLsp }); + try { + const result = await server.initialize(); + assert.ok(result.capabilities.textDocumentSync !== undefined, 'no textDocumentSync capability'); + const { code } = await server.stop(); + assert.equal(code, 0); + } finally { + server.child.kill('SIGKILL'); + } +}); + +scenario('2a. host, by name: runs on the host, RUST_LOG reaches it, a clean stop ends it', async () => { + await waitForNoHostServer('before'); + const { plan, launch, server } = await startAsExtension({ mode: 'host' }, { debug: true }); + try { + assert.equal(plan.kind, 'host'); + assert.equal(plan.program, 'comline-lsp'); + assert.deepEqual(launch.args.slice(0, 3), ['--host', '--watch-bus', '--env=RUST_LOG=debug']); + await server.initialize(); + const host = await describeHostServer(); + assert.equal(host.exe, config.hostLspUsr, 'the host server is not the host binary'); + assert.equal(host.rustLog, 'RUST_LOG=debug', 'RUST_LOG did not reach the host process'); + const { code } = await server.stop(); + assert.equal(code, 0); + } finally { + server.child.kill('SIGKILL'); + } + await waitForNoHostServer('after a clean stop'); +}); + +scenario('2b. host, customPath under the host-only /usr: runs, and no RUST_LOG when not debugging', async () => { + await waitForNoHostServer('before'); + const { plan, server } = await startAsExtension({ mode: 'host', customPath: config.hostLspUsr }); + try { + assert.equal(plan.kind, 'host'); + assert.equal(plan.program, config.hostLspUsr); + await server.initialize(); + const host = await describeHostServer(); + assert.equal(host.exe, config.hostLspUsr); + assert.equal(host.rustLog, undefined, `RUST_LOG is set without debug: ${host.rustLog}`); + await server.stop(); + } finally { + server.child.kill('SIGKILL'); + } + await waitForNoHostServer('after'); +}); + +scenario('2c. host, customPath in the home directory: killing flatpak-spawn stops the host server', async () => { + await waitForNoHostServer('before'); + const { server } = await startAsExtension({ mode: 'host', customPath: config.hostLspHome }); + await server.initialize(); + const host = await describeHostServer(); + assert.equal(host.exe, fs.realpathSync(config.hostLspHome)); + // SIGKILL: flatpak-spawn can't forward it or clean up. Only --watch-bus, + // the session helper noticing the bus connection close, can stop the server. + server.child.kill('SIGKILL'); + await waitForNoHostServer('after killing flatpak-spawn'); +}); + +scenario('3. host probe: ok, missing, and flatpak-spawn itself failing', async () => { + const run = async (program, env = process.env) => { + try { + const { stdout, stderr } = await execFileAsync(spawnPath, r.hostProbeArgs(program), { + timeout: 20000, + env, + }); + return r.interpretHostProbe({ exitCode: 0, stdout, stderr }); + } catch (e) { + return r.interpretHostProbe({ exitCode: e.code, stdout: e.stdout ?? '', stderr: e.stderr ?? '' }); + } + }; + assert.deepEqual(await run('comline-lsp'), { ok: true }); + assert.deepEqual(await run(config.hostLspHome), { ok: true }); + assert.deepEqual(await run('comline-no-such-program'), { ok: false, reason: 'missing' }); + assert.deepEqual(await run('/no/such/dir/comline-lsp'), { ok: false, reason: 'missing' }); + // No session bus to talk to: flatpak-spawn fails, and says why. + const failed = await run('comline-lsp', { + ...process.env, + DBUS_SESSION_BUS_ADDRESS: 'unix:path=/nonexistent/bus', + }); + assert.equal(failed.ok, false); + assert.equal(failed.reason, 'failed', JSON.stringify(failed)); + assert.notEqual(failed.detail, ''); + + // And what the extension shows for them. + const missing = await r.verifyPlan( + 'server', + r.planProgram({ tool: 'server', mode: 'host', customPath: '/no/such/dir/comline-lsp', bundled: null }, probe) + ); + assert.equal(missing.kind, 'error'); + assert.match(missing.message, /was not found on the host/); + assert.match(missing.message, /comline\.server\.customPath/); +}); + +scenario('4. path inside the sandbox is the host', async () => { + for (const [tool, program] of [ + ['server', 'comline-lsp'], + ['cli', 'comline'], + ]) { + const plan = r.planProgram({ tool, mode: 'path', customPath: '', bundled: null }, probe); + assert.deepEqual(plan, { kind: 'host', spawn: spawnPath, program }); + const verified = await r.verifyPlan(tool, plan); + assert.deepEqual(verified, plan, `${program} is not on the host: ${verified.message}`); + } +}); + +scenario('5. custom path in the host-only /usr: an error that points at host mode', async () => { + const plan = r.planProgram( + { tool: 'server', mode: 'custom', customPath: config.hostLspUsr, bundled: null }, + probe + ); + assert.equal(plan.kind, 'error'); + assert.match(plan.message, /comline\.server\.mode to 'host'/); + assert.match(plan.message, /\/usr/); + // The same file, by host mode, works (2b); here it is just out of reach. + const cli = r.planProgram({ tool: 'cli', mode: 'custom', customPath: '/usr/bin/comline', bundled: null }, probe); + assert.equal(cli.kind, 'error'); + assert.match(cli.message, /comline\.cli\.mode to 'host'/); +}); + +scenario('6. CLI in host mode: generation writes code on disk, generation clean removes it', async () => { + const pkg = config.package; + const generated = path.join(pkg, 'generated'); + fs.rmSync(generated, { recursive: true, force: true }); + + for (const mode of ['path', 'host']) { + const planned = r.planProgram({ tool: 'cli', mode, customPath: '', bundled: null }, probe); + const plan = await r.verifyPlan('cli', planned); + assert.equal(plan.kind, 'host', plan.message); + + // The launches runCliCommand and the status bar menu build. + const run = (subcommand) => { + const launch = r.launchFor(plan, ['--path', pkg, ...subcommand], { CLICOLOR_FORCE: '1' }, process.env); + assert.equal(launch.env, undefined); + return execFileAsync(launch.command, launch.args, { timeout: 120000 }); + }; + + const listing = r.launchFor(plan, ['targets'], {}, process.env); + const targets = await execFileAsync(listing.command, listing.args, { timeout: 60000 }); + assert.match(targets.stdout, /^\s*rust#/m, `comline targets: ${targets.stdout}`); + + await run(['generation']); + const files = fs.existsSync(generated) ? walk(generated) : []; + assert.ok(files.length > 0, `generation wrote nothing under ${generated}`); + assert.ok(files.every((f) => fs.statSync(f).size > 0), 'a generated file is empty'); + + await run(['generation', 'clean']); + assert.equal(fs.existsSync(generated), false, 'generation clean left the output behind'); + } + + // The old names are gone: what #30 was about. + const plan = r.planProgram({ tool: 'cli', mode: 'host', customPath: '', bundled: null }, probe); + const old = r.launchFor(plan, ['--path', pkg, 'generate'], {}, process.env); + await assert.rejects(execFileAsync(old.command, old.args, { timeout: 60000 })); +}); + +// -- Run ---------------------------------------------------------------- + +(async () => { + let failed = 0; + for (const { name, fn } of scenarios) { + const started = Date.now(); + try { + await withTimeout(fn(), 180000, `end of "${name}"`); + console.log(`ok ${name} (${Date.now() - started} ms)`); + } catch (e) { + failed += 1; + console.log(`FAIL ${name}`); + console.log(String(e && e.stack ? e.stack : e).replace(/^/gm, ' ')); + console.log(`::error title=Flatpak scenario failed::${name}: ${String(e && e.message).split('\n')[0]}`); + } + } + const total = scenarios.length; + console.log(`\n${total - failed}/${total} scenarios passed`); + process.exit(failed === 0 ? 0 : 1); +})(); diff --git a/src/extension.ts b/src/extension.ts index b724efa..881d7fd 100644 --- a/src/extension.ts +++ b/src/extension.ts @@ -47,7 +47,6 @@ import { import { Tool, Plan, - Probe, TOOLS, effectiveMode, planProgram, @@ -561,28 +560,6 @@ async function resolvePackageRoot(): Promise { return picked; } -/** What the resolution reads from this machine. */ -function systemProbe(): Probe { - return { - platform: process.platform, - env: process.env, - exists: (file) => fs.existsSync(file), - isExecutable: (file) => { - try { - if (!fs.statSync(file).isFile()) { - return false; - } - if (process.platform !== 'win32') { - fs.accessSync(file, fs.constants.X_OK); - } - return true; - } catch { - return false; - } - }, - }; -} - /** Where the language server or the CLI is and how to run it, from the * `comline.server.*` / `comline.cli.*` settings (see `resolve.ts`). A plan * that runs on the host through `flatpak-spawn` has been checked against the @@ -600,32 +577,7 @@ async function resolveProgram(tool: Tool, context?: ExtensionContext): Promise

fs.existsSync(file), + isExecutable: (file) => { + try { + if (!fs.statSync(file).isFile()) { + return false; + } + if (process.platform !== 'win32') { + fs.accessSync(file, fs.constants.X_OK); + } + return true; + } catch { + return false; + } + }, + }; +} + // -- PATH --------------------------------------------------------------- function pathApi(platform: NodeJS.Platform): path.PlatformPath { @@ -417,3 +446,36 @@ export function hostProbeMessage(tool: Tool, program: string, probe: HostProbe, `grants it).` ); } + +/** A plan, checked against the world: a `host` plan has to find the program + * on the host (and `flatpak-spawn` has to be allowed to reach it) before it is + * worth running; any other plan is returned as it is. Never throws: what went + * wrong is the `error` plan's message. */ +export async function verifyPlan( + tool: Tool, + plan: Plan, + env: NodeJS.ProcessEnv = process.env +): Promise { + if (plan.kind !== 'host') { + return plan; + } + let result: HostProbe; + try { + const { stdout, stderr } = await execFileAsync(plan.spawn, hostProbeArgs(plan.program), { + timeout: 10000, + env, + }); + result = interpretHostProbe({ exitCode: 0, stdout, stderr }); + } catch (err) { + const e = err as { code?: number | string; killed?: boolean; stdout?: string; stderr?: string; message?: string }; + result = interpretHostProbe({ + exitCode: e.killed ? 'timed out' : e.code, + stdout: e.stdout ?? '', + stderr: e.stderr ?? (e.code === undefined ? String(e.message ?? err) : ''), + }); + } + if (result.ok) { + return plan; + } + return { kind: 'error', message: hostProbeMessage(tool, plan.program, result, env.FLATPAK_ID) }; +} From e4856feefedc8a5ea690e26665f3bd0fee998cfd Mon Sep 17 00:00:00 2001 From: Kinflou Date: Mon, 16 Nov 2026 02:07:32 +0800 Subject: [PATCH 03/10] ci: shellcheck clean flatpak workflow --- .github/workflows/flatpak.yml | 26 ++++++++++++++------------ 1 file changed, 14 insertions(+), 12 deletions(-) diff --git a/.github/workflows/flatpak.yml b/.github/workflows/flatpak.yml index d012052..85d1c97 100644 --- a/.github/workflows/flatpak.yml +++ b/.github/workflows/flatpak.yml @@ -275,22 +275,23 @@ jobs: # A package to generate code from, made by the CLI itself. (cd "$HOME/e2e" && "$HOME/.local/bin/comline" new tinypkg && ls -R tinypkg) # What the test reads. - APP_REPO="$GITHUB_WORKSPACE" BUNDLED="$RUNNER_TEMP/bundled/comline-lsp" node -e ' - const home = process.env.HOME; - require("fs").writeFileSync(home + "/e2e/config.json", JSON.stringify({ - repo: process.env.APP_REPO, - bundledLsp: process.env.BUNDLED, - hostLspUsr: "/usr/local/bin/comline-lsp", - hostLspHome: home + "/e2e/hostonly/comline-lsp", - package: home + "/e2e/tinypkg", - appId: process.env.APP_ID, - }, null, 2));' + cat > "$HOME/e2e/config.json" <> "$GITHUB_OUTPUT" @@ -301,7 +302,8 @@ jobs: env: ELECTRON: ${{ steps.electron.outputs.bin }} run: | - export XDG_RUNTIME_DIR="/run/user/$(id -u)" + XDG_RUNTIME_DIR="/run/user/$(id -u)" + export XDG_RUNTIME_DIR # The host session's PATH, which `flatpak-spawn --host` programs get: # it is the environment the session bus is started in. export PATH="$HOME/.local/bin:$PATH" From 023e108187e4bc1dcf774817b75073781885a057 Mon Sep 17 00:00:00 2001 From: Kinflou Date: Mon, 16 Nov 2026 02:09:51 +0800 Subject: [PATCH 04/10] ci: shellcheck, the sandbox's own variable --- .github/workflows/flatpak.yml | 1 + 1 file changed, 1 insertion(+) diff --git a/.github/workflows/flatpak.yml b/.github/workflows/flatpak.yml index 85d1c97..acf1994 100644 --- a/.github/workflows/flatpak.yml +++ b/.github/workflows/flatpak.yml @@ -260,6 +260,7 @@ jobs: sudo mkdir -p "/run/user/$(id -u)" sudo chown "$(id -u):$(id -g)" "/run/user/$(id -u)" chmod 700 "/run/user/$(id -u)" + # shellcheck disable=SC2016 # $FLATPAK_ID is for the shell inside the sandbox XDG_RUNTIME_DIR="/run/user/$(id -u)" flatpak run --command=sh "$APP_ID" -c 'echo "inside the sandbox: $FLATPAK_ID"; cat /.flatpak-info | head -5' - name: Put the programs where the scenarios expect them From 393d0362ed63a0c1ca2ba3e1c070c8e91ac5ffbe Mon Sep 17 00:00:00 2001 From: Kinflou Date: Mon, 16 Nov 2026 02:13:26 +0800 Subject: [PATCH 05/10] Import the shared probe and host check --- src/extension.ts | 6 ++---- 1 file changed, 2 insertions(+), 4 deletions(-) diff --git a/src/extension.ts b/src/extension.ts index 881d7fd..2e443f8 100644 --- a/src/extension.ts +++ b/src/extension.ts @@ -51,10 +51,8 @@ import { effectiveMode, planProgram, launchFor, - hostProbeArgs, - interpretHostProbe, - HostProbe, - hostProbeMessage, + systemProbe, + verifyPlan, } from './resolve'; const execFileAsync = promisify(execFile); From bb686d665875603b9ddc10bd7608208c67fa56f8 Mon Sep 17 00:00:00 2001 From: Kinflou Date: Mon, 16 Nov 2026 02:18:17 +0800 Subject: [PATCH 06/10] ci: the app's Electron binary is com.vscodium.codium --- .github/workflows/flatpak.yml | 4 ++-- scripts/flatpak-e2e.cjs | 2 +- 2 files changed, 3 insertions(+), 3 deletions(-) diff --git a/.github/workflows/flatpak.yml b/.github/workflows/flatpak.yml index acf1994..f62aca8 100644 --- a/.github/workflows/flatpak.yml +++ b/.github/workflows/flatpak.yml @@ -294,8 +294,8 @@ jobs: XDG_RUNTIME_DIR="/run/user/$(id -u)" export XDG_RUNTIME_DIR flatpak run --command=sh "$APP_ID" -c 'ls -l /app/bin /app/share/codium | head -40' - flatpak run --command=sh "$APP_ID" -c 'test -x /app/share/codium/codium' - echo "bin=/app/share/codium/codium" >> "$GITHUB_OUTPUT" + flatpak run --command=sh "$APP_ID" -c 'test -x /app/share/codium/com.vscodium.codium' + echo "bin=/app/share/codium/com.vscodium.codium" >> "$GITHUB_OUTPUT" # -- the scenarios ------------------------------------------------------- diff --git a/scripts/flatpak-e2e.cjs b/scripts/flatpak-e2e.cjs index c41ce92..51152e8 100644 --- a/scripts/flatpak-e2e.cjs +++ b/scripts/flatpak-e2e.cjs @@ -2,7 +2,7 @@ // The extension's program resolution, run inside Flathub's real VSCodium // sandbox, against a real `flatpak-spawn --host`. // -// flatpak run --command=/app/share/codium/codium \ +// flatpak run --command=/app/share/codium/com.vscodium.codium \ // --env=ELECTRON_RUN_AS_NODE=1 com.vscodium.codium \ // scripts/flatpak-e2e.cjs // From 3d550c53c47c911605fdf352a57f687abb5b0d9e Mon Sep 17 00:00:00 2001 From: Kinflou Date: Mon, 16 Nov 2026 02:23:55 +0800 Subject: [PATCH 07/10] ci: prove --watch-bus with a process that ignores stdin, and a control --- scripts/flatpak-e2e.cjs | 36 ++++++++++++++++++++++++++++++++++++ 1 file changed, 36 insertions(+) diff --git a/scripts/flatpak-e2e.cjs b/scripts/flatpak-e2e.cjs index 51152e8..94dc746 100644 --- a/scripts/flatpak-e2e.cjs +++ b/scripts/flatpak-e2e.cjs @@ -276,6 +276,42 @@ scenario('2c. host, customPath in the home directory: killing flatpak-spawn stop await waitForNoHostServer('after killing flatpak-spawn'); }); +scenario('2d. --watch-bus is what ends a host process when flatpak-spawn dies (with a control)', async () => { + // comline-lsp also exits when its stdin closes, so 2c alone doesn't prove + // --watch-bus. `sleep` doesn't read stdin: only the bus watch can end it. + const plan = { kind: 'host', spawn: spawnPath, program: 'sleep' }; + const hostSleeps = async (seconds) => + (await onHost('pgrep -f "^sleep $1\\$" || true', String(seconds))).split('\n').filter((l) => l.trim()); + const waitFor = async (seconds, wanted, what) => { + const deadline = Date.now() + 15000; + for (;;) { + if ((await hostSleeps(seconds)).length > 0 === wanted) { + return; + } + assert.ok(Date.now() < deadline, `${what}: still ${wanted ? 'absent' : 'running'} after 15s`); + await sleep(250); + } + }; + + const watched = r.launchFor(plan, ['3141'], {}, process.env, { watchBus: true }); + const a = spawn(watched.command, watched.args, { stdio: ['pipe', 'ignore', 'ignore'] }); + await waitFor(3141, true, 'the watched sleep starting'); + a.kill('SIGKILL'); + await waitFor(3141, false, 'the watched sleep after killing flatpak-spawn'); + + // The control: the same without --watch-bus is left running. + const unwatched = r.launchFor(plan, ['3142'], {}, process.env, { watchBus: false }); + const b = spawn(unwatched.command, unwatched.args, { stdio: ['pipe', 'ignore', 'ignore'] }); + try { + await waitFor(3142, true, 'the unwatched sleep starting'); + b.kill('SIGKILL'); + await sleep(3000); + assert.equal((await hostSleeps(3142)).length, 1, 'without --watch-bus the host process should outlive flatpak-spawn'); + } finally { + await onHost('pkill -f "^sleep 3142$" || true'); + } +}); + scenario('3. host probe: ok, missing, and flatpak-spawn itself failing', async () => { const run = async (program, env = process.env) => { try { From 39d468f2d66de544680da1d5fb9e090e6a4e825f Mon Sep 17 00:00:00 2001 From: Kinflou Date: Mon, 16 Nov 2026 02:28:13 +0800 Subject: [PATCH 08/10] CHANGELOG: the Flatpak workflow --- CHANGELOG.md | 4 +++- 1 file changed, 3 insertions(+), 1 deletion(-) diff --git a/CHANGELOG.md b/CHANGELOG.md index 4a1a964..b6519f2 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -18,7 +18,9 @@ All notable changes to the "comline-vscode" extension will be documented in this `path`. `comline.server.customPath` / `comline.cli.customPath` name the program to run in `host` mode, for one that isn't on the host's `PATH` (`~/.cargo/bin`). -- Unit tests for the server and CLI resolution (`yarn test`), run in CI. +- Unit tests for the server and CLI resolution (`yarn test`), run in CI, and a + workflow that runs the same code inside Flathub's VSCodium sandbox against a + real `flatpak-spawn --host` (`scripts/flatpak-e2e.cjs`). - `comline.toml` is its own language, `comline-toml`, highlighted with a bundled TOML grammar (Taplo's, MIT; see `syntaxes/THIRD-PARTY-NOTICES.md`) and served by the Comline language server: diagnostics for its shape and From d7c3bf94eebdab791f010f582ee4df161e207ad7 Mon Sep 17 00:00:00 2001 From: Kinflou Date: Mon, 16 Nov 2026 03:16:21 +0800 Subject: [PATCH 09/10] ci: flatpak e2e on the ci-flatpak image, with the pinned language server built here --- .github/workflows/flatpak.yml | 279 +++++++++++++++++++--------------- CHANGELOG.md | 5 +- 2 files changed, 158 insertions(+), 126 deletions(-) diff --git a/.github/workflows/flatpak.yml b/.github/workflows/flatpak.yml index f62aca8..8e69391 100644 --- a/.github/workflows/flatpak.yml +++ b/.github/workflows/flatpak.yml @@ -17,27 +17,29 @@ name: Flatpak # 6. CLI `comline generation` and `generation clean`, through the # launch the extension builds, change the files on disk # -# What it needs, and why it is built this way: +# What it tests, and why it is built this way: # -# - A VM. Flatpak's sandbox (bubblewrap, user namespaces) does not run inside -# our CI container images. `ubuntu-24.04` is the VM the other workflows use. -# - `flatpak` from apt, at run time. EXCEPTION to the org's rule against -# installing packages while a job runs: the sandbox under test IS Flatpak, -# the runner image does not have it, and there is no pinned image that does. -# So the packages are pinned to exact versions (below) and the install is -# retried on network failure. Ubuntu keeps only its current security update -# of a package, so a version here goes stale when Ubuntu ships the next one: -# the weekly run turns that into a failed run that says which pin to move. -# - `com.vscodium.codium` from Flathub, user installation: the real app, with -# the permissions Flathub gives it, not a stand-in. It follows Flathub's -# current build, on purpose: a weekly run is how a change to its permissions -# (--talk-name=org.freedesktop.Flatpak, --filesystem=host) is noticed. The -# installed commit is logged, and the install is cached by that commit. -# - `dbus-run-session`: `flatpak-spawn --host` talks to the session helper -# over the session bus; a runner has none. -# - comline-lsp: the one the latest GitHub release ships (the linux-x64 -# package's bundled binary), not built again here. Which release is logged. -# - comline (CLI): built here from a pinned commit. It has no release yet. +# - The programs are this commit's. comline-lsp is built here from the +# language-server rev package.json pins (`comline.languageServer.rev`), the +# way release.yml builds the one the extension ships: static musl, in the +# tools image. comline (the CLI) is built from a pinned commit (it has no +# release yet). The sha256 of each is logged where it is used. +# - The sandbox is a container: ghcr.io/comlineproject/ci-flatpak (the +# ComlineProject/ci-images repository). It has Flatpak, bubblewrap, +# xdg-dbus-proxy and `dbus-run-session`, and Flathub's real +# `com.vscodium.codium` (the permissions Flathub gives it, not a stand-in), +# installed at an exact Flathub commit. Nothing is installed while the job +# runs. The VSCodium it ran on is logged; a new Flathub build is a new image +# (a digest bump), reviewed like any other change. +# - `--privileged`: bubblewrap makes namespaces and mounts /proc, which a +# container's default seccomp profile and masked paths forbid. +# - The container is the "host" of `flatpak-spawn --host`: the session helper +# runs in it, as `dbus-run-session` starts the session bus the helper needs +# (a runner has none). A path under /usr/local/bin is the host-only /usr: the +# sandbox has the runtime's /usr, so it does not see it. +# - The job runs as root, as every job container does (the workspace is mounted +# from the runner); flatpak needs no helper for that. The image's +# XDG_RUNTIME_DIR (/run/user/0) is the runtime directory flatpak wants. on: pull_request: @@ -56,7 +58,8 @@ on: - ".github/workflows/flatpak.yml" - "scripts/flatpak-e2e.cjs" schedule: - # Mondays: Flathub and Ubuntu move without a change here. + # Mondays: the runner's kernel, Docker and AppArmor move without a change + # here, and a sandbox that stops working on them should show up in a week. - cron: "23 5 * * 1" workflow_dispatch: @@ -77,13 +80,44 @@ env: # ComlineProject/cli, main when this was written. Bump with the CLI's # command set (it has to have `generation`, cli#153). CLI_REV: e79092a3cfec9f2b3d02a45189f51e6a90fb8071 - # Ubuntu 24.04 (noble), noble-updates/-security. - FLATPAK_VERSION: 1.14.6-1ubuntu0.1 - BUBBLEWRAP_VERSION: 0.9.0-1ubuntu0.3 - XDG_DBUS_PROXY_VERSION: 0.1.5-1ubuntu0.2 APP_ID: com.vscodium.codium jobs: + # out/resolve.js: the compiled module the test imports. Compiled on the + # runner (which has yarn), so the sandbox's container needs no Node. + compile: + name: Compile the extension + runs-on: ubuntu-24.04 + timeout-minutes: 15 + steps: + - name: Harden the runner + uses: step-security/harden-runner@1d88384fd173af6ecb012dfd0264a306b711241a # v2.22.4 + with: + egress-policy: audit + + - name: Checkout code + uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1 + with: + persist-credentials: false + + - name: Setup Node.js + uses: actions/setup-node@949feb2413d6458794dcd2491c4babbbce0c15c1 # v7.1.0 + with: + node-version: 20.20.2 + cache: yarn + + - name: Install dependencies + run: yarn install --frozen-lockfile + + - name: Compile TypeScript + run: yarn compile + + - uses: actions/upload-artifact@cf430e030ddbb5b0abf93d22962f4752f3646cd9 # v7.0.2 + with: + name: extension-out + path: out + if-no-files-found: error + cli: name: Build comline (CLI) runs-on: ubuntu-24.04 @@ -123,6 +157,7 @@ jobs: exit 1 fi "$bin" targets + sha256sum "$bin" - uses: actions/upload-artifact@cf430e030ddbb5b0abf93d22962f4752f3646cd9 # v7.0.2 with: @@ -130,36 +165,88 @@ jobs: path: target/x86_64-unknown-linux-musl/release/comline if-no-files-found: error - sandbox: - name: VSCodium sandbox (Flathub) - needs: cli + # The language server this commit would ship: the pin in package.json, built + # as release.yml's linux-x64 build does (the same image and target). + lsp: + name: Build comline-lsp (pinned) runs-on: ubuntu-24.04 - timeout-minutes: 45 + timeout-minutes: 60 + container: ghcr.io/comlineproject/ci-tools@sha256:b9301f9caf3e47f0b839de85efda178f698a55daf7fc387b463adeec93ecd433 steps: - - name: Harden the runner - uses: step-security/harden-runner@1d88384fd173af6ecb012dfd0264a306b711241a # v2.22.4 + - name: Checkout extension (for the pin) + uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1 with: - egress-policy: audit + persist-credentials: false - - name: Checkout code + - name: The pinned language-server rev + id: pin + run: | + rev=$(jq -r '.comline.languageServer.rev' package.json) + [[ "$rev" =~ ^[0-9a-f]{40}$ ]] || { echo "::error::no language-server rev in package.json: $rev"; exit 1; } + echo "language-server $rev" + echo "rev=$rev" >> "$GITHUB_OUTPUT" + + - name: Checkout language-server at the pinned rev uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1 with: + repository: ComlineProject/language-server + ref: ${{ steps.pin.outputs.rev }} + path: language-server persist-credentials: false - - name: Setup Node.js - uses: actions/setup-node@949feb2413d6458794dcd2491c4babbbce0c15c1 # v7.1.0 + - name: musl C compiler + run: | + # comline-core compiles its tree-sitter grammar from C at build time. + echo "CC_x86_64_unknown_linux_musl=musl-gcc" >> "$GITHUB_ENV" + + - uses: Swatinem/rust-cache@d1527c8f67ef9af357290475d4af4b0fab8f12e0 # v2.9.3 with: - node-version: 20.20.2 - cache: yarn + workspaces: language-server + key: x86_64-unknown-linux-musl - - name: Install dependencies - run: yarn install --frozen-lockfile + # The command release.yml runs for linux-x64. + - name: Build + working-directory: language-server + run: cargo build --release --target x86_64-unknown-linux-musl - # out/resolve.js: what the test imports. - - name: Compile TypeScript - run: yarn compile + - name: Static, and it runs + run: | + bin=language-server/target/x86_64-unknown-linux-musl/release/comline-lsp + if readelf -lW "$bin" | grep -q INTERP; then + echo "::error::comline-lsp is dynamically linked" + exit 1 + fi + sha256sum "$bin" + + - uses: actions/upload-artifact@cf430e030ddbb5b0abf93d22962f4752f3646cd9 # v7.0.2 + with: + name: comline-lsp + path: language-server/target/x86_64-unknown-linux-musl/release/comline-lsp + if-no-files-found: error + + sandbox: + name: VSCodium sandbox (Flathub) + needs: [compile, cli, lsp] + runs-on: ubuntu-24.04 + timeout-minutes: 45 + container: + # TODO(digest): the digest of the published ci-flatpak, from the + # ci-images `build` run on main (ComlineProject/ci-images#8). + image: ghcr.io/comlineproject/ci-flatpak@sha256:TODO + options: --privileged + steps: + - name: Checkout code + uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1 + with: + persist-credentials: false + + # -- what is under test --------------------------------------------------- - # -- the programs the extension will run ------------------------------- + - name: The compiled extension + uses: actions/download-artifact@9000827ccba6bdab643e8b6fd33ac0654aef8333 # v8.0.2 + with: + name: extension-out + path: out - name: The CLI (built above) uses: actions/download-artifact@9000827ccba6bdab643e8b6fd33ac0654aef8333 # v8.0.2 @@ -167,106 +254,52 @@ jobs: name: comline-cli path: ${{ runner.temp }}/cli - - name: The language server (the latest release's bundled one) - env: - GH_TOKEN: ${{ github.token }} + - name: The language server (built above, from the pinned rev) + uses: actions/download-artifact@9000827ccba6bdab643e8b6fd33ac0654aef8333 # v8.0.2 + with: + name: comline-lsp + path: ${{ runner.temp }}/bundled + + # Static: checked where each was built (a static binary is what runs in + # the sandbox, whose libraries are the runtime's; scenario 1 runs it). + - name: The programs, as built run: | - tag=$(gh release list --repo "$GITHUB_REPOSITORY" --limit 1 --json tagName --jq '.[0].tagName') - echo "language server from release $tag" - gh release download "$tag" --repo "$GITHUB_REPOSITORY" \ - --pattern '*-linux-x64.vsix' --dir "$RUNNER_TEMP/release" - mkdir -p "$RUNNER_TEMP/bundled" - unzip -p "$RUNNER_TEMP"/release/*-linux-x64.vsix extension/bin/linux-x64/comline-lsp \ - > "$RUNNER_TEMP/bundled/comline-lsp" - chmod +x "$RUNNER_TEMP/bundled/comline-lsp" - sha256sum "$RUNNER_TEMP/bundled/comline-lsp" - # The bundled server is static, which is why it runs in a sandbox. - if readelf -lW "$RUNNER_TEMP/bundled/comline-lsp" | grep -q INTERP; then - echo "::error::the release's comline-lsp is dynamically linked" - exit 1 - fi + chmod +x "$RUNNER_TEMP/bundled/comline-lsp" "$RUNNER_TEMP/cli/comline" + sha256sum "$RUNNER_TEMP/bundled/comline-lsp" "$RUNNER_TEMP/cli/comline" # -- Flatpak ------------------------------------------------------------- - # EXCEPTION to "no packages installed while a job runs": see the header. - # Exact versions, retried on a network failure. - - name: Install Flatpak (apt, pinned) + - name: Which image, which VSCodium run: | - retry() { - for attempt in 1 2 3 4; do - "$@" && return 0 - echo "attempt $attempt failed: $*" >&2 - sleep $((attempt * 10)) - done - return 1 - } - retry sudo apt-get update -q - retry sudo apt-get install -y -q --no-install-recommends \ - "flatpak=$FLATPAK_VERSION" \ - "bubblewrap=$BUBBLEWRAP_VERSION" \ - "xdg-dbus-proxy=$XDG_DBUS_PROXY_VERSION" + echo "VSCodium commit: $(cat /etc/ci-flatpak/vscodium-commit)" + echo "runtime commit: $(cat /etc/ci-flatpak/runtime-commit)" flatpak --version - command -v dbus-run-session - - - name: Flathub, and which build of the app - id: app - run: | - flatpak remote-add --user --if-not-exists flathub https://dl.flathub.org/repo/flathub.flatpakrepo - commit=$(flatpak remote-info --user --show-commit flathub "$APP_ID") - echo "Flathub's current $APP_ID commit: $commit" - echo "commit=$commit" >> "$GITHUB_OUTPUT" - - # Keyed on the app's commit: most runs find it and download nothing. A new - # Flathub build misses, falls back to the latest older cache (the - # runtimes are shared), and updates it. - - name: Flatpak cache - id: cache - uses: actions/cache@55cc8345863c7cc4c66a329aec7e433d2d1c52a9 # v6.1.0 - with: - path: ~/.local/share/flatpak - key: flatpak-${{ env.FLATPAK_VERSION }}-${{ env.APP_ID }}-${{ steps.app.outputs.commit }} - restore-keys: | - flatpak-${{ env.FLATPAK_VERSION }}-${{ env.APP_ID }}- - - - name: Install the app - run: | - flatpak remote-add --user --if-not-exists flathub https://dl.flathub.org/repo/flathub.flatpakrepo - echo "cache hit: ${{ steps.cache.outputs.cache-hit }}" - if [ "${{ steps.cache.outputs.cache-hit }}" = true ]; then - echo "restored from the cache: nothing to download" - else - flatpak install --user --noninteractive --assumeyes flathub "$APP_ID" - # A restored older build is brought up to date. - flatpak update --user --noninteractive --assumeyes "$APP_ID" - fi - flatpak info --user --show-commit "$APP_ID" - flatpak info --user --show-permissions "$APP_ID" - du -sh ~/.local/share/flatpak + flatpak list --system --columns=ref,active,origin,installation + [ "$(flatpak info --system --show-commit "$APP_ID")" = "$(cat /etc/ci-flatpak/vscodium-commit)" ] # The documented facts the extension relies on, as the installed app has # them: if Flathub drops either permission this is where it shows. - name: The app has the permissions host mode needs run: | - permissions=$(flatpak info --user --show-permissions "$APP_ID") + permissions=$(flatpak info --system --show-permissions "$APP_ID") grep -E '^filesystems=.*\bhost\b' <<<"$permissions" grep -E '^org\.freedesktop\.Flatpak=talk' <<<"$permissions" # -- the session --------------------------------------------------------- - - name: Can this runner run a Flatpak sandbox at all? + - name: Can this container run a Flatpak sandbox at all? run: | sysctl kernel.apparmor_restrict_unprivileged_userns kernel.unprivileged_userns_clone 2>&1 || true - # flatpak wants a runtime dir, which a runner has no login session to make. - sudo mkdir -p "/run/user/$(id -u)" - sudo chown "$(id -u):$(id -g)" "/run/user/$(id -u)" - chmod 700 "/run/user/$(id -u)" + echo "XDG_RUNTIME_DIR=$XDG_RUNTIME_DIR" + test -d "$XDG_RUNTIME_DIR" # shellcheck disable=SC2016 # $FLATPAK_ID is for the shell inside the sandbox - XDG_RUNTIME_DIR="/run/user/$(id -u)" flatpak run --command=sh "$APP_ID" -c 'echo "inside the sandbox: $FLATPAK_ID"; cat /.flatpak-info | head -5' + flatpak run --command=sh "$APP_ID" -c 'echo "inside the sandbox: $FLATPAK_ID"; cat /.flatpak-info | head -5' - name: Put the programs where the scenarios expect them run: | - # The host-only /usr: a file the sandbox cannot see at that path. - sudo install -m 755 "$RUNNER_TEMP/bundled/comline-lsp" /usr/local/bin/comline-lsp + # The host-only /usr: a file the sandbox cannot see at that path (its + # /usr is the runtime's). The container is the host here. + install -m 755 "$RUNNER_TEMP/bundled/comline-lsp" /usr/local/bin/comline-lsp # A path under the home directory that is on no PATH. mkdir -p "$HOME/e2e/hostonly" install -m 755 "$RUNNER_TEMP/bundled/comline-lsp" "$HOME/e2e/hostonly/comline-lsp" @@ -291,8 +324,6 @@ jobs: - name: Find the app's Electron binary id: electron run: | - XDG_RUNTIME_DIR="/run/user/$(id -u)" - export XDG_RUNTIME_DIR flatpak run --command=sh "$APP_ID" -c 'ls -l /app/bin /app/share/codium | head -40' flatpak run --command=sh "$APP_ID" -c 'test -x /app/share/codium/com.vscodium.codium' echo "bin=/app/share/codium/com.vscodium.codium" >> "$GITHUB_OUTPUT" @@ -303,8 +334,6 @@ jobs: env: ELECTRON: ${{ steps.electron.outputs.bin }} run: | - XDG_RUNTIME_DIR="/run/user/$(id -u)" - export XDG_RUNTIME_DIR # The host session's PATH, which `flatpak-spawn --host` programs get: # it is the environment the session bus is started in. export PATH="$HOME/.local/bin:$PATH" diff --git a/CHANGELOG.md b/CHANGELOG.md index b6519f2..25173c6 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -20,7 +20,10 @@ All notable changes to the "comline-vscode" extension will be documented in this (`~/.cargo/bin`). - Unit tests for the server and CLI resolution (`yarn test`), run in CI, and a workflow that runs the same code inside Flathub's VSCodium sandbox against a - real `flatpak-spawn --host` (`scripts/flatpak-e2e.cjs`). + real `flatpak-spawn --host` (`scripts/flatpak-e2e.cjs`), with the language + server built from the rev `package.json` pins and the CLI from a pinned + commit, in the `ci-flatpak` image (Flatpak and VSCodium at an exact Flathub + commit, nothing installed while the job runs). - `comline.toml` is its own language, `comline-toml`, highlighted with a bundled TOML grammar (Taplo's, MIT; see `syntaxes/THIRD-PARTY-NOTICES.md`) and served by the Comline language server: diagnostics for its shape and From ebdb18fc2ba769aed52f74d7524b9c90e719197e Mon Sep 17 00:00:00 2001 From: Kinflou Date: Mon, 16 Nov 2026 03:49:42 +0800 Subject: [PATCH 10/10] ci: pin the published ci-flatpak image --- .github/workflows/flatpak.yml | 4 +--- 1 file changed, 1 insertion(+), 3 deletions(-) diff --git a/.github/workflows/flatpak.yml b/.github/workflows/flatpak.yml index 8e69391..11c9875 100644 --- a/.github/workflows/flatpak.yml +++ b/.github/workflows/flatpak.yml @@ -230,9 +230,7 @@ jobs: runs-on: ubuntu-24.04 timeout-minutes: 45 container: - # TODO(digest): the digest of the published ci-flatpak, from the - # ci-images `build` run on main (ComlineProject/ci-images#8). - image: ghcr.io/comlineproject/ci-flatpak@sha256:TODO + image: ghcr.io/comlineproject/ci-flatpak@sha256:664637fe0967ad5e01f47e9356fdd941bd2fd03b3e4ab268fcc37aef3916e0a7 options: --privileged steps: - name: Checkout code