fix: add request timeouts and mask API key in repr#1262
Conversation
Add explicit timeouts to two previously unbounded requests calls, and mask the API key in OpenAIBackend.__repr__ / __str__. Timeouts -------- RESTHandler.emit() uses timeout=2: this handler fires synchronously on every log event, so a slow or unreachable webhook endpoint would block the calling thread on every log write. 2 s is enough to confirm liveness; beyond that the record should be dropped. is_vllm_server_with_structured_output() uses timeout=10: this probe runs once at OpenAIBackend.__init__() time, not on every request, so a longer timeout costs nothing in throughput. vLLM's /version endpoint is trivial (no GPU path), but cloud or VPN round-trips can be slow; 10 s covers realistic slow-network cases while still failing fast on a dead server. A false timeout here silently degrades structured output behaviour, so erring toward patience is correct. API key masking --------------- __repr__ and __str__ previously inherited the default object repr, which could expose the API key in logs, exception messages, or debug output. Both now return *** in place of the key. Identified in the 2026-05-12 security audit. Closes generative-computing#1246 Assisted-by: Claude Code Signed-off-by: Nigel Jones <jonesn@uk.ibm.com>
e8ed118 to
465f56f
Compare
|
Awaiting review - any chance you could take a look @AngeloDanducci |
|
Will take a look today 👍 |
AngeloDanducci
left a comment
There was a problem hiding this comment.
Looks mostly good - couple of tests to be added just to cover all the bases I think.
Add unit tests for RESTHandler.emit() timeout parameter and OpenAIBackend repr/str masking, as requested by reviewer. Also fix repr to show None rather than '***' when api_key is not explicitly set (key comes from env). Assisted-by: Claude Code Signed-off-by: Nigel Jones <jonesn@uk.ibm.com>
AngeloDanducci
left a comment
There was a problem hiding this comment.
LGTM feedback addressed.
|
@nrfulton @jakelorocco — this PR has been open for 7 days and has one approval. It needs a sign-off from a CODEOWNER of |
jakelorocco
left a comment
There was a problem hiding this comment.
The timeout defaults seem reasonable to me.
@planetf1, I don't understand this comment:
`repr` and `str` previously inherited the default object repr,
which could expose the API key in logs, exception messages, or debug output.
Both now return `***` in place of the key.
Identified in the 2026-05-12 security audit.
For objects that don't define str, it's my understanding that Python falls back to the repr method. The default repr method for Backends should just be <mellea.backends.openai.OpenAIBackend object at 0x101db7920>. I don't understand where the credential leaking is coming from. I do agree though: we could have nicer output for backends. But we should then probably handle all of the backends.
|
@jakelorocco Fair point — the default repr is just The motivation was more "give it a useful repr that's safe by default" — if someone later adds attributes or converts to a dataclass, the key's already masked from the start. I'll update the description to say that. Agreed on the other backends — filed #1317 to cover them. |
7d47f52
Add explicit timeouts to two previously unbounded
requestscalls, andmask the API key in
OpenAIBackend.__repr__/__str__.Timeouts
RESTHandler.emit()usestimeout=2: this handler fires synchronouslyon every log event, so a slow or unreachable webhook endpoint would block
the calling thread on every log write. 2 s is enough to confirm liveness;
beyond that the record should be dropped.
is_vllm_server_with_structured_output()usestimeout=10: this proberuns once at
OpenAIBackend.__init__()time, not on every request, so alonger timeout costs nothing in throughput. vLLM's
/versionendpoint istrivial (no GPU path), but cloud or VPN round-trips can be slow; 10 s covers
realistic slow-network cases while still failing fast on a dead server. A
false timeout here silently degrades structured output behaviour, so erring
toward patience is correct.
API key masking
__repr__and__str__previously fell back to Python's default object repr — safe, but not very useful. Both now return a structured repr showingmodel_idandbase_url, with the API key replaced by***. The masking is defensive: if the repr is later extended, the key is already guarded.Identified in the 2026-05-12 security audit. Ref #1246. Follow-up to cover other backends: #1317.