Skip to content

fix: enforce configurable per-model image input limits #3562

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
wsp1911 merged 2 commits into from
Oct 27, 2026
Merged

fix: enforce configurable per-model image input limits #3562

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
2 commits
Select commit Hold shift + click to select a range
389950a
fix: enforce configurable per-model image input limits
uiharuayako Oct 27, 2026
1bf6962
fix: include translation dependency in image intake effect
uiharuayako Oct 27, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions OpenBitFun-Installer/src-tauri/src/installer/ai_config.rs
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -45,6 +45,7 @@ pub(super) fn ai_config_from_installer_model(m: &ModelConfig) -> Result<AIConfig
format: format_key.to_string(),
context_window: 128_128,
max_tokens: None,
max_images_per_request: None,
temperature: None,
top_p: None,
inline_think_in_text: false,
Expand Down
3 changes: 3 additions & 0 deletions src/apps/cli/src/model_selection.rs
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -95,6 +95,7 @@ pub(crate) fn model_from_mutation(
api_key,
context_window: mutation.context_window,
max_tokens: mutation.max_tokens,
max_images_per_request: current.max_images_per_request,
temperature: current.temperature,
top_p: current.top_p,
enabled: mutation.enabled,
Expand Down Expand Up @@ -198,12 +199,14 @@ mod tests {
.unwrap();
let existing = openbitfun_core::service::config::AIModelConfig {
api_key: "synthetic-existing".into(),
max_images_per_request: Some(3),
channel_routing: Some(serde_json::from_value(serde_json::json!({"version":1,"fallback_model_ids":["backup"],"manual_channel":"backup"})).unwrap()),
..Default::default()
};
let preserved = model_from_mutation(mutation, Some(existing.clone())).unwrap();
assert_eq!(preserved.channel_routing, existing.channel_routing);
assert_eq!(preserved.api_key, existing.api_key);
assert_eq!(preserved.max_images_per_request, Some(3));
assert_eq!(preserved.name, "Edited");
}

Expand Down
18 changes: 18 additions & 0 deletions src/apps/desktop/README.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -714,6 +714,24 @@ scheduled-job and desktop-result notifications are unavailable there. Ordinary
WeChat-initiated requests keep their existing cross-device reply path. This feature
does not add cross-host Peer Device or Detached Dispatch result delivery.

## Image input limits

For a vision model that accepts only a limited number of images, set **Images
per request** in its model settings (for example, `3`). Leave it empty to
keep the existing image-history behavior. The saved model field is
`max_images_per_request`; it limits input images, including user attachments,
retained history, and tool results, rather than generated images.

User submissions above the configured limit are rejected before entering the
conversation. Pools and channel routes use the smallest configured limit among
their enabled vision models when admitting new attachments, so fallback models
can accept the same batch. Each model request keeps the newest image-bearing
messages within that limit and includes an omission notice for pixels left out. The Agent can
read omitted files again in smaller batches; the original conversation is retained.
Plain text continuation also applies the limit, including to old tool-image history.
Each selected model or fallback channel uses its own limit at the executing host;
older hosts must be upgraded to enforce this optional setting.

## Model pools

In **Settings → AI models → Model pools**, create named pools from existing model
Expand Down
6 changes: 6 additions & 0 deletions src/crates/adapters/ai-adapters/AGENTS.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -36,6 +36,12 @@ provider-neutral contracts owned by `openbitfun-agent-stream`.
`cargo test --locked -p openbitfun-ai-adapters --lib providers::openai::chat::tests`.
- Do not move provider-neutral stream DTOs, replay policy, or tool-call
accumulation ownership back into this crate.
- Optional model image-input ceilings are applied to outgoing message copies at
the shared AIClient send boundary, including single-attempt Responses calls.
Count user image parts and tool attachments together; preserve durable history,
tool-result identities, opaque tool text and non-image media. Newer messages
keep their pixels first, with explicit omission text for older pixels. Verify
with `cargo test --locked -p openbitfun-ai-adapters --lib client::image_budget::tests`.
- Normalize explicitly reported response models into the typed stream
`response_model` observation. Responses/Codex lifecycle metadata is emitted
before an associated error, and Chat Completions model identifiers remain
Expand Down
49 changes: 33 additions & 16 deletions src/crates/adapters/ai-adapters/src/client.rs
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -7,6 +7,7 @@
pub(crate) mod format;
pub(crate) mod healthcheck;
pub(crate) mod http;
mod image_budget;
#[cfg(feature = "image-generation")]
pub mod image_generation;
pub(crate) mod quirks;
Expand Down Expand Up @@ -401,22 +402,6 @@ impl AIClient {
trace: Option<ModelExchangeTraceConfig>,
) -> Result<StreamResponse> {
let custom_body = self.config.custom_request_body.clone();
if matches!(
ApiFormat::parse(&self.config.format)?,
ApiFormat::OpenAIResponses
) {
return openai::responses::send_stream(
self,
messages,
tools,
custom_body,
1,
trace,
request_context,
)
.await;
}

self.send_message_stream_with_extra_body_and_max_attempts(
messages,
tools,
Expand Down Expand Up @@ -611,6 +596,10 @@ impl AIClient {
trace: Option<ModelExchangeTraceConfig>,
request_context: Option<ModelRequestContext>,
) -> Result<StreamResponse> {
// Project only the outgoing copy, against the actual selected client.
// Account/channel failover and direct analysis calls share this boundary.
let messages = image_budget::project(messages, self.config.max_images_per_request)?;
image_budget::validate_extra_body(self.config.max_images_per_request, extra_body.as_ref())?;
match ApiFormat::parse(&self.config.format)? {
ApiFormat::OpenAIChat => {
openai::chat::send_stream(
Expand Down Expand Up @@ -871,6 +860,7 @@ mod tests {
format: format.to_string(),
context_window: 128000,
max_tokens: Some(8192),
max_images_per_request: None,
temperature: None,
top_p: None,
inline_think_in_text: false,
Expand Down Expand Up @@ -1335,6 +1325,7 @@ mod tests {
format: "openai".to_string(),
context_window: 128000,
max_tokens: Some(8192),
max_images_per_request: None,
temperature: None,
top_p: None,
inline_think_in_text: false,
Expand Down Expand Up @@ -1368,6 +1359,7 @@ mod tests {
format: "anthropic".to_string(),
context_window: 200000,
max_tokens: Some(8192),
max_images_per_request: None,
temperature: None,
top_p: None,
inline_think_in_text: false,
Expand Down Expand Up @@ -1400,6 +1392,7 @@ mod tests {
format: "gemini".to_string(),
context_window: 128000,
max_tokens: Some(4096),
max_images_per_request: None,
temperature: Some(0.2),
top_p: Some(0.8),
inline_think_in_text: false,
Expand Down Expand Up @@ -1482,6 +1475,7 @@ mod tests {
format: "gemini".to_string(),
context_window: 128000,
max_tokens: Some(4096),
max_images_per_request: None,
temperature: None,
top_p: None,
inline_think_in_text: false,
Expand Down Expand Up @@ -1529,6 +1523,7 @@ mod tests {
format: "openai".to_string(),
context_window: 128000,
max_tokens: Some(4096),
max_images_per_request: None,
temperature: None,
top_p: None,
inline_think_in_text: false,
Expand Down Expand Up @@ -1566,6 +1561,7 @@ mod tests {
format: "openai".to_string(),
context_window: 128000,
max_tokens: Some(4096),
max_images_per_request: None,
temperature: None,
top_p: None,
inline_think_in_text: false,
Expand Down Expand Up @@ -1605,6 +1601,7 @@ mod tests {
format: "openai".to_string(),
context_window: 128000,
max_tokens: Some(4096),
max_images_per_request: None,
temperature: None,
top_p: None,
inline_think_in_text: false,
Expand Down Expand Up @@ -1642,6 +1639,7 @@ mod tests {
format: "openai".to_string(),
context_window: 1_000_000,
max_tokens: Some(8192),
max_images_per_request: None,
temperature: None,
top_p: None,
inline_think_in_text: false,
Expand Down Expand Up @@ -1683,6 +1681,7 @@ mod tests {
format: "openai".to_string(),
context_window: 1_000_000,
max_tokens: Some(8192),
max_images_per_request: None,
temperature: None,
top_p: None,
inline_think_in_text: false,
Expand Down Expand Up @@ -1860,6 +1859,7 @@ mod tests {
format: "openai".to_string(),
context_window: 128_000,
max_tokens: Some(4096),
max_images_per_request: None,
temperature: None,
top_p: None,
inline_think_in_text: false,
Expand Down Expand Up @@ -1899,6 +1899,7 @@ mod tests {
format: "openai".to_string(),
context_window: 128000,
max_tokens: Some(4096),
max_images_per_request: None,
temperature: None,
top_p: None,
inline_think_in_text: false,
Expand Down Expand Up @@ -1936,6 +1937,7 @@ mod tests {
format: "responses".to_string(),
context_window: 128000,
max_tokens: Some(4096),
max_images_per_request: None,
temperature: None,
top_p: None,
inline_think_in_text: false,
Expand Down Expand Up @@ -1977,6 +1979,7 @@ mod tests {
format: "anthropic".to_string(),
context_window: 200000,
max_tokens: Some(8192),
max_images_per_request: None,
temperature: None,
top_p: None,
inline_think_in_text: false,
Expand Down Expand Up @@ -2017,6 +2020,7 @@ mod tests {
format: "anthropic".to_string(),
context_window: 200000,
max_tokens: Some(8192),
max_images_per_request: None,
temperature: None,
top_p: None,
inline_think_in_text: false,
Expand Down Expand Up @@ -2056,6 +2060,7 @@ mod tests {
format: "anthropic".to_string(),
context_window: 200000,
max_tokens: Some(8192),
max_images_per_request: None,
temperature: None,
top_p: None,
inline_think_in_text: false,
Expand Down Expand Up @@ -2095,6 +2100,7 @@ mod tests {
format: "anthropic".to_string(),
context_window: 200000,
max_tokens: Some(8192),
max_images_per_request: None,
temperature: None,
top_p: None,
inline_think_in_text: false,
Expand Down Expand Up @@ -2136,6 +2142,7 @@ mod tests {
format: "anthropic".to_string(),
context_window: 200000,
max_tokens: Some(8192),
max_images_per_request: None,
temperature: None,
top_p: None,
inline_think_in_text: false,
Expand Down Expand Up @@ -2170,6 +2177,7 @@ mod tests {
format: "anthropic".to_string(),
context_window: 200000,
max_tokens: Some(8192),
max_images_per_request: None,
temperature: None,
top_p: None,
inline_think_in_text: false,
Expand Down Expand Up @@ -2211,6 +2219,7 @@ mod tests {
format: "anthropic".to_string(),
context_window: 1_000_000,
max_tokens: Some(8192),
max_images_per_request: None,
temperature: None,
top_p: None,
inline_think_in_text: false,
Expand Down Expand Up @@ -2253,6 +2262,7 @@ mod tests {
format: "anthropic".to_string(),
context_window: 1_000_000,
max_tokens: Some(8192),
max_images_per_request: None,
temperature: None,
top_p: None,
inline_think_in_text: false,
Expand Down Expand Up @@ -2293,6 +2303,7 @@ mod tests {
format: "anthropic".to_string(),
context_window: 1_000_000,
max_tokens: Some(8192),
max_images_per_request: None,
temperature: None,
top_p: None,
inline_think_in_text: false,
Expand Down Expand Up @@ -2331,6 +2342,7 @@ mod tests {
format: "anthropic".to_string(),
context_window: 1_000_000,
max_tokens: Some(8192),
max_images_per_request: None,
temperature: None,
top_p: None,
inline_think_in_text: false,
Expand Down Expand Up @@ -2371,6 +2383,7 @@ mod tests {
format: "anthropic".to_string(),
context_window: 1_000_000,
max_tokens: Some(8192),
max_images_per_request: None,
temperature: None,
top_p: None,
inline_think_in_text: false,
Expand Down Expand Up @@ -2413,6 +2426,7 @@ mod tests {
format: "anthropic".to_string(),
context_window: 200000,
max_tokens: Some(4000),
max_images_per_request: None,
temperature: None,
top_p: None,
inline_think_in_text: false,
Expand Down Expand Up @@ -2451,6 +2465,7 @@ mod tests {
format: "anthropic".to_string(),
context_window: 200000,
max_tokens: Some(8192),
max_images_per_request: None,
temperature: None,
top_p: None,
inline_think_in_text: false,
Expand Down Expand Up @@ -2485,6 +2500,7 @@ mod tests {
format: "anthropic".to_string(),
context_window: 200000,
max_tokens: Some(8192),
max_images_per_request: None,
temperature: None,
top_p: None,
inline_think_in_text: false,
Expand Down Expand Up @@ -2523,6 +2539,7 @@ mod tests {
format: "anthropic".to_string(),
context_window: 200000,
max_tokens: Some(8192),
max_images_per_request: None,
temperature: None,
top_p: None,
inline_think_in_text: false,
Expand Down
Loading
Loading