Hosted model definitions instead of waiting for new releases

[Sarke]

Why does every change in models or pricing need a new PR and new release? E.g. #1755 and #1756

It would be nice if this was just a simple json file for each provider with model definitions. It could live in a separate repo that can be updated much more regularly and hosted on github pages and/or some other CDN like CloudFlare. Then cache it locally for an hour or even 24 hours with a simple button to download the latest definitions.

The model definitions are basically json already.

Then you can have another repo with scripts that automatically hit the respective providers' APIs for updates, such as https://api.openai.com/v1/models or https://openrouter.ai/api/v1/models, after which it will push changes to the repo with just the json model definitions. Or do it manually, which would still be much easier than currently, where each change requires a new PR.

You can even do schema versioning if backwards compatibility is a concern, so a schema change wouldn't break current/older versions.

[Sarke]

It even says it does for openai, but I don't think so. I don't see gpt-6-sol or gpt-6-luna. I mean, why else would #1755 be needed? They're half-price too...

[edelauna]

The /models endpoint generally advertises model names, there’s no standardized model metadata so knowing context limit and pricing is usually why a PR is required in zoo

[Sarke]

Yeah, but even so that's why I suggest a separate repo to host those definitions, since they can easily and quickly be updated (either manually or automatically) without having to mess with the main repo code.

And you wouldn't have to wait to push a new release to the extension marketplace, to get the new definitions.

The /models endpoint generally advertises model names, there’s no standardized model metadata so knowing context limit and pricing is usually why a PR is required in zoo

Either way, you need to compile the data.

[edelauna]

could maybe look at model.dev or liteLLM endpoint. But would still need to map models to providers which would still like require a PR.