Repository navigation
Reject unparseable GUI click responses instead of inventing origin clicks - #1722
Merged
ssiq merged 3 commits intoOct 14, 2026
Merged
Conversation
…icks Signed-off-by: Rudy Celekli <rudy@gradiahq.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
GUI click parsers return
[0.0, 0.0]when the response contains no complete click. A failed or empty model response can therefore count as a correct click when the target contains the origin, while Format_Err_Rate remains zero. Raise ValueError for these responses so the existing evaluator exception handlers record a format failure. A genuinepyautogui.click(x=0, y=0)remains valid.Apply the same correction to the duplicated parsers in ScreenSpot, ScreenSpot-Pro, ScreenSpot-v2, OSWorld-G, VBGD, and VenusBench. Coordinate normalization is unchanged.
Validation: tests cover all six parsers and the real ScreenSpot point evaluator with origin targets. Accuracy and format error rate now both report 50% for half valid and half empty responses, instead of 100% accuracy and 0% format errors. Tests fail on main and pass with this change;
pre-commit run --all-filespasses. Image loading is stubbed; no model inference was performed.AI disclosure: this change and its regression tests were prepared with assistance from OpenAI Codex.