Repository navigation
Zenodo call throws error due to request rate limit #117
Description
Activity
Not cool, but fair enough. The actual culprit should be this limit:
Request frequency: Maximum 30 requests per minute
Right? So could a simple
time.sleep(2)[2 secs] before therequestcall solve this issue? To max out the rate limit, I could imagine a tracker of the current request rate, so that the sleep time could be adapted accordingly. Maybe not necessary.
How many single request do the test suite and tutorials perform?Reacted by Pablo Iturrieta- linked a pull request that will close this issue117 zenodo call throws error due to request rate limit #118
on Jan 23, 2026 Thank you Marcus for checking! This issue came from a long time, in which also I had to "skip" the test of this tutorial in the CI.
The main culprits were:
- If models came from the same repo, the first model downloaded everything, and then the other models skipped the download (doesnt make too much sense).
- Having too many simultaneous requests, such as getting all files from a repo at once and get the HEAD (i.e., content size) to provide status bars. This was acceptable for "local" runs in the past, but the CI was always doing multiple requests from all py versions/os, so I skipped CI tests. However, Zenodo reduced the request rate to 30/min, so now even local fails.
I implemented your suggestions, reduced the amount of tutorial d) files, skipped all requests that were meant to get the HEAD and handled some server errors. Those changes were included in #118, and works okay now :D! THank you
Reacted by Marcus HerrmannGreat solution! It looks very adaptive to me and generally optimized :-)
I'm not sure, but I think you don't have to drop progress bars, since additionally requesting the HEAD was only supposed to get the total file size, correct?
I wonder if you could simplify
download_file()further by passing the file size, which is already returned with the initial request. You can get it with:download_sizes = [f["size"] for f in files]Just an idea.
Zenodo requests causes error in request json parsing
floatcsep/floatcsep/utils/accessors.py
Line 23 in 5b3e00f
This is caused by a HTML reply rather than a JSON, due to Zenodo limiting
rate requests (https://blog.zenodo.org/2025/11/25/2025-11-14-search-api-updates/). This causes tests and tutorials to usually fail. This might be due to multiple model
flavourscalls in the same zenodo repository in quick succession, which label the IP to be causing suspicious traffic.