Skip to content

fix: add request timeouts and mask API key in repr#1262

Merged
planetf1 merged 2 commits into
generative-computing:mainfrom
planetf1:fix/security-hardening
Jun 23, 2026
Merged

fix: add request timeouts and mask API key in repr#1262
planetf1 merged 2 commits into
generative-computing:mainfrom
planetf1:fix/security-hardening

Conversation

@planetf1

@planetf1 planetf1 commented Jun 12, 2026

Copy link
Copy Markdown
Contributor

Add explicit timeouts to two previously unbounded requests calls, and
mask the API key in OpenAIBackend.__repr__ / __str__.

Timeouts

RESTHandler.emit() uses timeout=2: this handler fires synchronously
on every log event, so a slow or unreachable webhook endpoint would block
the calling thread on every log write. 2 s is enough to confirm liveness;
beyond that the record should be dropped.

is_vllm_server_with_structured_output() uses timeout=10: this probe
runs once at OpenAIBackend.__init__() time, not on every request, so a
longer timeout costs nothing in throughput. vLLM's /version endpoint is
trivial (no GPU path), but cloud or VPN round-trips can be slow; 10 s covers
realistic slow-network cases while still failing fast on a dead server. A
false timeout here silently degrades structured output behaviour, so erring
toward patience is correct.

API key masking

__repr__ and __str__ previously fell back to Python's default object repr — safe, but not very useful. Both now return a structured repr showing model_id and base_url, with the API key replaced by ***. The masking is defensive: if the repr is later extended, the key is already guarded.

Identified in the 2026-05-12 security audit. Ref #1246. Follow-up to cover other backends: #1317.

@github-actions github-actions Bot added the bug Something isn't working label Jun 12, 2026
@planetf1 planetf1 marked this pull request as ready for review June 12, 2026 11:39
@planetf1 planetf1 requested review from a team, jakelorocco and nrfulton as code owners June 12, 2026 11:39
@planetf1 planetf1 requested a review from AngeloDanducci June 12, 2026 11:39
Add explicit timeouts to two previously unbounded requests calls, and
mask the API key in OpenAIBackend.__repr__ / __str__.

Timeouts
--------
RESTHandler.emit() uses timeout=2: this handler fires synchronously on
every log event, so a slow or unreachable webhook endpoint would block
the calling thread on every log write. 2 s is enough to confirm
liveness; beyond that the record should be dropped.

is_vllm_server_with_structured_output() uses timeout=10: this probe
runs once at OpenAIBackend.__init__() time, not on every request, so
a longer timeout costs nothing in throughput. vLLM's /version endpoint
is trivial (no GPU path), but cloud or VPN round-trips can be slow;
10 s covers realistic slow-network cases while still failing fast on a
dead server. A false timeout here silently degrades structured output
behaviour, so erring toward patience is correct.

API key masking
---------------
__repr__ and __str__ previously inherited the default object repr,
which could expose the API key in logs, exception messages, or debug
output. Both now return *** in place of the key.

Identified in the 2026-05-12 security audit.

Closes generative-computing#1246

Assisted-by: Claude Code
Signed-off-by: Nigel Jones <jonesn@uk.ibm.com>
@planetf1 planetf1 force-pushed the fix/security-hardening branch from e8ed118 to 465f56f Compare June 12, 2026 11:57
@planetf1

Copy link
Copy Markdown
Contributor Author

Awaiting review - any chance you could take a look @AngeloDanducci

@AngeloDanducci

Copy link
Copy Markdown
Contributor

Will take a look today 👍

@AngeloDanducci AngeloDanducci left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks mostly good - couple of tests to be added just to cover all the bases I think.

Comment thread mellea/backends/openai.py
Comment thread mellea/core/utils.py
Add unit tests for RESTHandler.emit() timeout parameter and OpenAIBackend
repr/str masking, as requested by reviewer. Also fix repr to show None
rather than '***' when api_key is not explicitly set (key comes from env).

Assisted-by: Claude Code
Signed-off-by: Nigel Jones <jonesn@uk.ibm.com>

@AngeloDanducci AngeloDanducci left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM feedback addressed.

@planetf1 planetf1 enabled auto-merge June 17, 2026 07:51
@planetf1

Copy link
Copy Markdown
Contributor Author

@nrfulton @jakelorocco — this PR has been open for 7 days and has one approval. It needs a sign-off from a CODEOWNER of mellea/core/ (which is the two of you). Would appreciate a look when you can.

@jakelorocco jakelorocco left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The timeout defaults seem reasonable to me.

@planetf1, I don't understand this comment:

`repr` and `str` previously inherited the default object repr,
which could expose the API key in logs, exception messages, or debug output.
Both now return `***` in place of the key.

Identified in the 2026-05-12 security audit.

For objects that don't define str, it's my understanding that Python falls back to the repr method. The default repr method for Backends should just be <mellea.backends.openai.OpenAIBackend object at 0x101db7920>. I don't understand where the credential leaking is coming from. I do agree though: we could have nicer output for backends. But we should then probably handle all of the backends.

@planetf1

Copy link
Copy Markdown
Contributor Author

@jakelorocco Fair point — the default repr is just <ClassName object at 0x...>, nothing leaks.

The motivation was more "give it a useful repr that's safe by default" — if someone later adds attributes or converts to a dataclass, the key's already masked from the start. I'll update the description to say that.

Agreed on the other backends — filed #1317 to cover them.

@planetf1 planetf1 requested a review from jakelorocco June 23, 2026 09:43
@planetf1 planetf1 added this pull request to the merge queue Jun 23, 2026
Merged via the queue into generative-computing:main with commit 7d47f52 Jun 23, 2026
18 of 19 checks passed
@planetf1 planetf1 deleted the fix/security-hardening branch June 23, 2026 14:53
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

bug Something isn't working

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants