Is it possible to see through openai library how many maximum tokens can be in a model?

[Zapzatron]

Is it possible to see through openai library or other way how many maximum tokens can be in a model, for example gpt-4? I need to find out the maximum number of tokens in a model using python code.

For example, gpt-4 has a maximum of 8000 tokens, how can I get this number from the python code?

[rattrayalex]

cc @logankilpatrick

[Zapzatron]

Can you drop a link of this?

[GautamVhavle]

The Python SDK does not know the context window. client.models.retrieve("gpt-4") returns id, created, owned_by, object. There is no max_tokens / context_length field on that object, and models.list() is the same.

Token limits live in model docs, not in the API. Anything you "get from Python" is a table you maintain (or a library that maintains one). LiteLLM, openai-model-registry, and similar packages hardcode this. They go stale.

Practical options:

from openai import OpenAI
client = OpenAI()
model = client.models.retrieve("gpt-4")  # no context length here

• For prompt size vs a known limit, count with tiktoken (cl100k_base for gpt-4/gpt-4o families, o200k_base for gpt-4o and 4.1) and compare to a constant you set.
• After a call, response.usage is what you actually spent. It still does not tell you the cap.
• Do not scrape the 8k figure for "gpt-4". That id has been 8k and 32k (gpt-4-32k) and is not what current apps should target anyway. Pin a dated snapshot (gpt-4o-2024-08-06) and keep the window next to it in config.

There is no supported client.models.context_length("gpt-4").