Skip to content

Validate top-p sampling bounds - #790

Open
sylvesterkaczmarek wants to merge 2 commits into
google-deepmind:mainfrom
sylvesterkaczmarek:fix/topp-sampling-p-validation
Open

Validate top-p sampling bounds#790
sylvesterkaczmarek wants to merge 2 commits into
google-deepmind:mainfrom
sylvesterkaczmarek:fix/topp-sampling-p-validation

Conversation

@sylvesterkaczmarek

Copy link
Copy Markdown

Summary

TopPSampling.get_next_tokens() currently accepts p values outside the probability range. Values below 0 collapse the cutoff to the highest-logit token, while values above 1 skip nucleus filtering entirely, so invalid configuration silently changes sampling behaviour.

This validates p before sampling and raises a clear ValueError unless it is within [0, 1]. Boundary values remain supported: p=0 keeps the current top-1 behaviour and p=1 keeps unrestricted sampling.

Testing

  • Added parametrized regression coverage for p=-0.1 and p=1.1.
  • Existing top-p tests remain unchanged.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant