Skip to content

Fix #1: let Ollama thinking models pass the local readiness probe - #17

Open
xiaoyao-666-coder wants to merge 2 commits into
zhengdaodeguL:mainfrom
xiaoyao-666-coder:fix/issue-1-ollama-reasoning
Open

xiaoyao-666-coder wants to merge 2 commits into
zhengdaodeguL:mainfrom
xiaoyao-666-coder:fix/issue-1-ollama-reasoning

Conversation

@xiaoyao-666-coder

Copy link
Copy Markdown

Closes #1.

MEDGUIDE_MODE=local 下连接 Ollama 时,_local_request_options() 固定发送 llama.cpp 专用的 chat_template_kwargs.enable_thinking=false,在 Ollama + gemma4:e2b 上没有关闭思考,16 token 的 readiness probe 全部用于 reasoning,message.content 为空,模型被判为不可用。

  • 新增 MEDGUIDE_LOCAL_LLM_PROVIDER:默认 llama.cpp(保持现有行为);设为 ollama 时 probe 与正式生成请求都改发 reasoning_effort="none"。非法值在构造时即抛 ValueError,不静默用错配置;非 local 模式下该变量被忽略。
  • SDK 隔离测试参数化为 llama.cpp / ollama(含大小写与空格)两种情况,核对 probe 与 generate 的实际请求正文;新增 5 个非法值测试。README 与本地模型文档补了说明。

验证:MEDGUIDE_MODE=test python -m pytest -q backend/tests 529 passed(基线 521 + 8 新);前端未改动。

说明:本机没有 Ollama,reasoning_effort="none" 能让 gemma4:e2b 返回 content "OK" 是 issue 报告人的实测结论,本次只验证了发出的请求正文正确,未在真实 Ollama 0.33.1 上跑过 probe。

…ess probe

Add MEDGUIDE_LOCAL_LLM_PROVIDER (llama.cpp by default, ollama opt-in).
With ollama, _local_request_options sends reasoning_effort='none'
instead of the llama.cpp-only chat_template_kwargs.enable_thinking
flag, covering both the readiness probe and generate requests; the
flag previously left reasoning on, so a 16-token probe was consumed
by reasoning and returned empty content, marking a usable model
unavailable. Invalid values fail fast at startup. Docs updated.

Closes zhengdaodeguL#1.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

本地模式兼容性:Ollama 思考模型的 readiness probe 因未关闭 reasoning 返回空正文

1 participant