PAIR version or commit
0.91.7 (nvpair-engine-manager 0.17.4, linux/arm64). The code paths below are also present on main.
Affected component
Go services — nvpair-engine-manager
Environment
NVIDIA DGX Spark (GB10, aarch64), Ubuntu 24.04.4 LTS, driver 580.173.02, 10-node PAIR cluster. Reproduced against stock binaries with no local modifications.
Steps to reproduce
- Configure an engine other than Ollama.
- Issue an
engine:action whose HTTP call needs more than 30 seconds to return response headers. The realistic case is a chat action against a large model with a long prompt, where prefill alone can exceed 30s before the first header is written.
- Observe the action fail with a transport timeout.
Expected behavior
The per-action timeout should be a property of the action or the manifest rather than of the engine's name, so an engine author can declare that a given action may be slow.
Actual behavior
executor.go defines two clients:
engineResponseHeaderTimeout = 30 * time.Second
ollamaLoadResponseHeaderTimeout = 10 * time.Minute
and actions.go chooses between them with a literal engine-name comparison:
client := e.client
if engine == "ollama" && action == "run_model" && e.ollamaLoadClient != nil {
client = e.ollamaLoadClient
}Every other engine therefore gets 30 seconds for every action, and no manifest field can change it.
By code inspection this is not limited to third-party engines. The shipped lmstudio.json declares a chat action (POST /v1/chat/completions) that uses the 30s client, and the vLLM manifest proposed in #9 declares a chat action as well, so a first-class vLLM engine would inherit the same ceiling.
The failure also presents badly: net/http: timeout awaiting response headers reads as a broken or hung engine rather than a client-side limit.
Sanitized logs or screenshots
action "run_model": Post "http://127.0.0.1:8800/v1/chat/completions":
net/http: timeout awaiting response headers [30s]
Scope of what I actually observed, to be precise: I hit this with an engine registered through a user manifest in <appdir>/engines/, driving engine-manager over its stdio JSON-RPC. It reproduced consistently while the model was cold and stopped once I pre-warmed the model during load, which is what isolated it to the response-header timeout rather than to the engine itself. I have not reproduced a 30s chat failure on LM Studio directly; that exposure is read from its manifest and the selection logic above.
Suggested fix
An optional per-action timeout_s in the manifest, defaulting to the current 30s, would cover this and would let the Ollama exception move into ollama.json instead of living in Go — removing the engine-name comparison entirely.
PAIR version or commit
0.91.7 (nvpair-engine-manager 0.17.4, linux/arm64). The code paths below are also present on
main.Affected component
Go services — nvpair-engine-manager
Environment
NVIDIA DGX Spark (GB10, aarch64), Ubuntu 24.04.4 LTS, driver 580.173.02, 10-node PAIR cluster. Reproduced against stock binaries with no local modifications.
Steps to reproduce
engine:actionwhose HTTP call needs more than 30 seconds to return response headers. The realistic case is achataction against a large model with a long prompt, where prefill alone can exceed 30s before the first header is written.Expected behavior
The per-action timeout should be a property of the action or the manifest rather than of the engine's name, so an engine author can declare that a given action may be slow.
Actual behavior
executor.godefines two clients:and
actions.gochooses between them with a literal engine-name comparison:Every other engine therefore gets 30 seconds for every action, and no manifest field can change it.
By code inspection this is not limited to third-party engines. The shipped
lmstudio.jsondeclares achataction (POST /v1/chat/completions) that uses the 30s client, and the vLLM manifest proposed in #9 declares achataction as well, so a first-class vLLM engine would inherit the same ceiling.The failure also presents badly:
net/http: timeout awaiting response headersreads as a broken or hung engine rather than a client-side limit.Sanitized logs or screenshots
Scope of what I actually observed, to be precise: I hit this with an engine registered through a user manifest in
<appdir>/engines/, driving engine-manager over its stdio JSON-RPC. It reproduced consistently while the model was cold and stopped once I pre-warmed the model during load, which is what isolated it to the response-header timeout rather than to the engine itself. I have not reproduced a 30schatfailure on LM Studio directly; that exposure is read from its manifest and the selection logic above.Suggested fix
An optional per-action
timeout_sin the manifest, defaulting to the current 30s, would cover this and would let the Ollama exception move intoollama.jsoninstead of living in Go — removing the engine-name comparison entirely.