Repository navigation
fix(copilot): cap tool calls per turn and widen the openai pin - #843
Conversation
Two follow-ups from the review on hyperledger-cello#837. max_iterations counts model rounds, not work. A single round can ask for any number of tools, so the rounds cap on its own does not bound how much a turn does. CELLO_COPILOT_MAX_TOOL_CALLS bounds the calls themselves, defaulting to 16. The check runs before a call rather than after, so the cap is the number executed. Hitting it ends the turn with stop_reason max_tool_calls, which is a new value in the set the done frame may carry. The turn has to end there rather than skip the remaining calls: a provider rejects a follow-up request whose assistant message has tool calls without a reply for each one. Ending means no follow-up request is made, so the conversation stays consistent. openai was pinned exactly at 1.99.9. Widened to >=1.55.3,<2 so dependabot can take minor bumps. The floor is unchanged and still load bearing, since earlier releases pass proxies= to httpx.Client, which httpx 0.28 removed. Worth knowing that openai is now on 3.x, so the ceiling holds us on the 1.x line until someone checks the newer client surface against this code. Signed-off-by: Aghamatlab Akbarzade <icloudakberzade@gmail.com>
|
Any PR that rebuilds api-engine will fail the same way. #842 migrates |
yeasy
left a comment
There was a problem hiding this comment.
LGTM, thanks for following up on #837. The cap is checked before each call, so it bounds what actually runs, and ending the turn there keeps the conversation consistent. The two new tests cover both sides.
One non-blocking note on the openai pin: I don't see a .github/dependabot.yml, so dependabot only opens security updates here. Moving to >=1.55.3,<2 mostly means each image build picks up whatever the latest 1.x is, and builds are no longer reproducible. I'd lean toward keeping an exact pin (bumped to the latest 1.x) and opening a separate issue to evaluate moving to 3.x with a real test run, as you suggested. Happy to take that in a follow-up rather than block this PR.
Follow-ups 2 and 3 from @yeasy's review on #837. Item 4 adds a route so it is
separate, and item 1 is waiting on the question I asked there.
Tool call cap
max_iterationscounts model rounds. One round can ask for any number oftools, so the rounds cap on its own does not bound how much work a turn does.
Eight rounds of five parallel calls is forty REST requests against a cap that
says eight.
CELLO_COPILOT_MAX_TOOL_CALLSbounds the calls themselves, default 16. Thecheck runs before a call rather than after, so the cap is the number executed.
Hitting it ends the turn with
stop_reason: max_tool_calls. That is a new valuein the set the
doneframe can carry, so I have added it to the contract in#813. @Pranav-d33 it is additive and the reply text still arrives normally, so
the panel needs no change, but worth knowing it exists.
Ending the turn is deliberate rather than skipping the rest of the round. A
provider rejects a follow-up request whose assistant message contains tool calls
with no reply for each one. Ending means no follow-up request happens, so the
conversation never goes inconsistent.
openai pin
openai==1.99.9becomesopenai>=1.55.3,<2so dependabot can take minor bumps.The floor stays and is still load bearing: earlier releases pass
proxies=tohttpx.Client, which httpx 0.28 removed, so building a client raisesTypeError.One thing worth flagging. openai is on 3.x now, so
<2holds us on the 1.x linerather than just guarding against one future major. I kept it because nobody has
checked the 2.x or 3.x client surface against this code, and a silent major bump
in a dependabot PR is the wrong place to find out. Happy to raise the ceiling in
its own PR if you would rather, since that wants a real test run.
Testing
37 tests pass locally, two of them new: one drives rounds of three parallel
calls against a cap of four and asserts four ran and the turn ended with
max_tool_calls, the other checks a turn under the cap is untouched.Docker will not start on my machine at the moment, so the endpoint tests ran
against a minimal Django config rather than the real project settings. CI runs
them properly.
I also pinned my local openai to the allowed range before testing, so the run
reflects what CI installs rather than whatever was already there.