Repository navigation
fix(lib): only raise finish-reason errors when there is parseable input #3589
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
base: main
Are you sure you want to change the base?
Changes from all commits
File filter
Filter by extension
Conversations
Jump to
Diff view
Diff view
There are no files selected for viewing
| Original file line number | Diff line number | Diff line change |
|---|---|---|
|
|
@@ -104,12 +104,20 @@ def parse_chat_completion( | |
| else: | ||
| input_tools = [] | ||
|
|
||
| # `length` / `content_filter` finish reasons only prevent us from producing a valid | ||
| # parsed result when there is actually something to parse. For a plain completion | ||
| # (no `response_format` and no parseable tools) there is nothing to parse, so we | ||
| # mirror the streaming accumulator (`ChatCompletionStreamState`), which guards these | ||
| # errors with `has_parseable_input`, and leave the completion untouched — matching | ||
| # `chat.completions.create()`. | ||
| raise_on_incomplete = has_parseable_input(response_format=response_format, input_tools=input_tools) | ||
|
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more.
When Useful? React with 👍 / 👎. |
||
|
|
||
| choices: list[ParsedChoice[ResponseFormatT]] = [] | ||
| for choice in chat_completion.choices: | ||
| if choice.finish_reason == "length": | ||
| if raise_on_incomplete and choice.finish_reason == "length": | ||
| raise LengthFinishReasonError(completion=chat_completion) | ||
|
|
||
| if choice.finish_reason == "content_filter": | ||
| if raise_on_incomplete and choice.finish_reason == "content_filter": | ||
| raise ContentFilterFinishReasonError(completion=chat_completion) | ||
|
|
||
| message = choice.message | ||
|
|
||
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
When
toolsis a generator (which the publicIterableannotation permits), both sync and asyncchat.completions.parse()first consume it in_validate_input_tools(completions.py:179andcompletions.py:1790) and then pass the exhausted iterator here. Consequentlyinput_toolsbecomes empty andraise_on_incompleteis false, so a request that supplied a strict parseable tool but finishes withlengthorcontent_filternow silently returns an incomplete completion instead of raising. Materialize the tools before validation or otherwise preserve their parseability state.Useful? React with 👍 / 👎.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This was an older bug on this branch's base: the generator was used up before the request body was built, so the tool was never sent. #3770 (33ffa1f) fixed it on main by turning tools into a list before validation. This PR is now rebased onto main (226630f), and a generator
tools=withfinish_reason=lengthraisesLengthFinishReasonErrorfor sync and async.parse()and.stream()(checked locally).