From 63dc9bf88431a9ed13e2c8ab907377b80a5727ee Mon Sep 17 00:00:00 2001 From: Pas <74743263+Pasithea0@users.noreply.github.com> Date: Sat, 17 Oct 2026 14:14:46 -0600 Subject: [PATCH 1/3] feat: move to TheIntroDB API v4 with TVDB-first lookups The API's v4 surface keys every response by tvdb_id and treats TVDB as the media's identity. Ask with tvdb_id first (tmdb_id and imdb_id still resolve), and read the identity from the response instead of assuming tmdb_id. Episodes send their own TVDB episode id when Kodi has one, which pins the exact episode record instead of trusting the numbering, plus the series' display order when it is one the API can convert. A 404 now falls through to the next identity rather than giving up: a lookup addressed only by tmdb_id is read as TMDB numbering, so an episode stored under another order is answered 'not found' while the same episode resolves by TVDB id (issue #19 - 'rarely works' on streams whose numbering follows the release). A 429 or an unreachable host still stops the chain, since another id would not be answered either. Key validation moves to /v4/user/stats. It also used to probe a media lookup, which cannot tell an invalid key from a media with no data, so an invalid key was reported as valid. v2 and v3 no longer serve /user/stats at all. 429 handling now reads the limit's own headers: Retry-After first, the daily budget's reset trusted up to 24h and a burst reset clamped to 5 minutes, instead of trusting whichever header came first. Adds tests/ with unit coverage of the request shape, the fallback order, the 429 header semantics and the key probe. --- README.md | 12 +- plugin.video.tidb/introdb.py | 300 +++++++++++++++++++++++++++-------- plugin.video.tidb/player.py | 93 +++++++++++ plugin.video.tidb/service.py | 10 +- tests/test_introdb_v4.py | 212 +++++++++++++++++++++++++ 5 files changed, 557 insertions(+), 70 deletions(-) create mode 100644 tests/test_introdb_v4.py diff --git a/README.md b/README.md index b20ddf0..d089068 100644 --- a/README.md +++ b/README.md @@ -6,9 +6,9 @@ Kodi service add-on that gets intro, recap, credits, and preview segments from **[TheIntroDB](https://theintrodb.org)** for movies and TV shows and shows a skip button or auto skips for you! -**Requirements:** Kodi 19+. **TMDb metadata is recommended** for best accuracy. IMDb works as a fallback for supported items. +**Requirements:** Kodi 19+. **TVDB metadata is recommended**: TVDB is the API's identity, so it is what the add-on asks with first (TMDB, then IMDb, are fallbacks). -**Important:** Lookups happen when playback starts. If the current item does not expose a usable **TMDb** or **IMDb** ID, the add-on cannot match it with TheIntroDB. +**Important:** Lookups happen when playback starts. If the current item does not expose a usable **TVDB**, **TMDb** or **IMDb** ID, the add-on cannot match it with TheIntroDB. **Troubleshooting (no skip button):** See the Metadata Requirements and Installation sections below. @@ -39,13 +39,17 @@ Kodi service add-on that gets intro, recap, credits, and preview segments from * ### Metadata Requirements -**TMDB is recommended.** The add-on prefers TMDB IDs for matching. If TMDB is unavailable, it can fall back to IMDb `tt...` IDs. +**TVDB is recommended.** TVDB is TheIntroDB's identity, so it is what the add-on asks with first; **TMDB** is next, then **IMDb** `tt...`. When the item exposes several, the add-on uses the best one and falls back to the others if the first finds nothing. + +Alongside the id, an episode's own **TVDB episode id** is sent when Kodi has one, which pins the exact episode instead of trusting the numbering. For TV episodes, the playing item also needs valid **season** and **episode** numbers. If your source add-on or library does not expose provider IDs, TheIntroDB cannot match the item. +Known limitation: for a series whose episode numbers follow a release order rather than TVDB's default (streaming rips, absolute-numbered anime, some specials), a lookup addressed only by **TMDB** can miss content that is stored under another numbering. Kodi items carrying a TVDB id resolve exactly; for the others, passing season/episode as the release numbers them is all the add-on can do. + ## How It Works -On each new playback, the add-on reads what Kodi exposes for the current item through JSON-RPC `Player.GetItem` and `VideoInfoTag`: **TMDb ID** when available, otherwise **IMDb** `tt...` ID, plus **season** and **episode** for TV content. +On each new playback, the add-on reads what Kodi exposes for the current item through JSON-RPC `Player.GetItem` and `VideoInfoTag`: **TVDB ID** when available, otherwise **TMDb ID**, otherwise **IMDb** `tt...` ID, plus **season**, **episode** and (for episodes) the **TVDB episode ID**. It then calls TheIntroDB, retrieves segment **start** and **end** times, waits until the segment window begins, and either shows the skip overlay or seeks automatically depending on your settings. diff --git a/plugin.video.tidb/introdb.py b/plugin.video.tidb/introdb.py index 1992581..9417a43 100644 --- a/plugin.video.tidb/introdb.py +++ b/plugin.video.tidb/introdb.py @@ -14,11 +14,22 @@ except ImportError: from urllib2 import Request, urlopen, HTTPError, URLError +from urllib.parse import quote + ADDON = xbmcaddon.Addon() _ADDON_ID = ADDON.getAddonInfo('id') -API_BASE = 'https://api.theintrodb.org/v3' +USER_AGENT = 'TheIntroDB Kodi Addon/1.0' + +API_BASE = 'https://api.theintrodb.org/v4' +API_VERSION = 'v4' MIN_REQUEST_GAP = 0.4 # small gap between requests +# A burst 429 resets within the 10s window; a daily-budget 429 carries seconds +# until UTC midnight (up to 86400) and must be trusted that far, or an exhausted +# bucket becomes a probe-every-five-minutes loop. +RATE_RESET_CLAMP = 5 * 60 +USAGE_RESET_CLAMP = 24 * 60 * 60 +DEFAULT_RETRY_SECONDS = 5 * 60 _last_request_time = 0.0 _rate_limit_until = 0.0 _rate_limit_lock = threading.Lock() @@ -42,16 +53,21 @@ def _get_api_key() -> str: def test_api_key() -> Tuple[bool, str]: """Validate the configured API key against the TheIntroDB API. + Probes /user/stats, the one endpoint whose answer depends on the credential + alone: a media lookup cannot distinguish a bad key from a media that simply + has no data, so it reported an invalid key as valid. v2 and v3 no longer + serve /user/stats (they are frozen at GET /media), so this speaks v4. + Returns (success, message) tuple. """ api_key = _get_api_key() if not api_key: return False, 'No API key configured.' - url = '{}/media?tmdb_id=1'.format(API_BASE) + url = '{}/user/stats'.format(API_BASE) req = Request(url) req.add_header('Accept', 'application/json') - req.add_header('User-Agent', 'TheIntroDB Kodi Addon/1.0') + req.add_header('User-Agent', USER_AGENT) req.add_header('Authorization', 'Bearer {}'.format(api_key)) try: @@ -66,7 +82,7 @@ def test_api_key() -> Tuple[bool, str]: return False, 'Rate limited (429). Try again later.' if e.code >= 500: return False, 'API server error (HTTP {}). Try again later.'.format(e.code) - return True, 'API key is valid.' + return False, 'API key could not be verified (HTTP {}).'.format(e.code) except URLError as e: return False, 'API unreachable: {}'.format(e.reason) except Exception as e: @@ -95,11 +111,64 @@ def _wait_rate_limit() -> bool: return True -def _do_request(url: str, api_key: str) -> Optional[Dict[str, Any]]: +def _header_int(headers: Any, name: str) -> Optional[int]: + if headers is None: + return None + try: + raw = headers.get(name) + except Exception: + return None + if raw is None: + return None + try: + return int(float(str(raw).strip())) + except (TypeError, ValueError): + return None + + +def _is_usage_limit_body(body: str) -> bool: + if not body: + return False + try: + code = json.loads(body).get('code') + except (ValueError, TypeError): + return False + return code in ('usage_limit_exceeded', 'specific_media_usage_limit_exceeded') + + +def _retry_after_seconds(error: Any, body: str) -> int: + """Seconds to wait after a 429, from whichever signal is trustworthy.""" + headers = getattr(error, 'headers', None) + is_usage_limit = _is_usage_limit_body(body) + usage_reset = _header_int(headers, 'X-UsageLimit-Reset') + if usage_reset and usage_reset > RATE_RESET_CLAMP: + # A reset beyond the burst window is unambiguous even when an + # intermediary stripped the body. + is_usage_limit = True + + retry = _header_int(headers, 'Retry-After') + if not retry: + retry = usage_reset + if not retry: + retry = _header_int(headers, 'X-RateLimit-Reset') + if not retry: + retry = DEFAULT_RETRY_SECONDS + + ceiling = USAGE_RESET_CLAMP if is_usage_limit else RATE_RESET_CLAMP + return max(1, min(retry, ceiling)) + + +def _do_request(url: str, api_key: str) -> Tuple[str, Optional[Dict[str, Any]]]: + """Perform a GET, returning (status, data). + + Status is 'ok', 'not_found', 'rate_limited' or 'error'. The caller needs the + distinction: a 404 means this addressing found nothing and another id worth + trying may, whereas a 429 or a transport error means nothing was answered. + """ global _rate_limit_until req = Request(url) req.add_header('Accept', 'application/json') - req.add_header('User-Agent', 'TheIntroDB Kodi Addon/1.0') + req.add_header('User-Agent', USER_AGENT) if api_key: req.add_header('Authorization', 'Bearer {}'.format(api_key)) @@ -108,35 +177,33 @@ def _do_request(url: str, api_key: str) -> Optional[Dict[str, Any]]: body = resp.read().decode('utf-8') data = json.loads(body) _log_resp(body) - return data + return 'ok', data except HTTPError as e: + if e.code == 404: + xbmc.log('[TheIntroDB] TheIntroDB 404: {} not in database'.format(url), + xbmc.LOGINFO) + return 'not_found', None if e.code == 429: - retry = 300 - for header in ('X-UsageLimit-Reset', 'X-RateLimit-Reset', 'Retry-After'): - val = e.headers.get(header) - if val: - try: - retry = int(val) - except ValueError: - pass - break + try: + err_body = e.read().decode('utf-8') + except Exception: + err_body = '' + retry = _retry_after_seconds(e, err_body) with _rate_limit_lock: _rate_limit_until = time.time() + retry - xbmc.log('[TheIntroDB] TheIntroDB 429 rate limited for {}s'.format(retry), + xbmc.log('[TheIntroDB] TheIntroDB 429 limit reached, pausing {}s'.format(retry), xbmc.LOGWARNING) - elif e.code == 404: - xbmc.log('[TheIntroDB] TheIntroDB 404: not in database', xbmc.LOGINFO) - else: - xbmc.log('[TheIntroDB] TheIntroDB HTTP {}'.format(e.code), xbmc.LOGWARNING) - return None + return 'rate_limited', None + xbmc.log('[TheIntroDB] TheIntroDB HTTP {}'.format(e.code), xbmc.LOGWARNING) + return 'error', None except URLError as e: xbmc.log('[TheIntroDB] TheIntroDB network error: {}'.format(e.reason), xbmc.LOGWARNING) - return None + return 'error', None except Exception as e: xbmc.log('[TheIntroDB] TheIntroDB request failed: {}'.format(e), xbmc.LOGERROR) - return None + return 'error', None def _pick_best_segment(segments: List[Dict[str, Any]]) -> Tuple[Optional[float], Optional[float]]: @@ -262,6 +329,13 @@ def _valid_tmdb(tmdb_id: Optional[Union[str, int]]) -> bool: return False +def _valid_numeric_id(value: Optional[Union[str, int]]) -> bool: + try: + return int(str(value)) > 0 + except (ValueError, TypeError): + return False + + def _episode_nums(season: Optional[Union[str, int]], episode: Optional[Union[str, int]]) -> Tuple[Optional[int], Optional[int]]: try: s = int(season) @@ -271,8 +345,23 @@ def _episode_nums(season: Optional[Union[str, int]], episode: Optional[Union[str return None, None -def _build_url(tmdb_id: Optional[Union[str, int]], imdb_id: Optional[str], season: Optional[Union[str, int]], episode: Optional[Union[str, int]], is_movie: bool, duration_ms: Optional[Union[str, int]] = None) -> Tuple[Optional[str], Optional[str]]: - # prefer tmdb; if missing use imdb (api matches show/episode) +def _build_url(tvdb_id: Optional[Union[str, int]] = None, + tmdb_id: Optional[Union[str, int]] = None, + imdb_id: Optional[str] = None, + season: Optional[Union[str, int]] = None, + episode: Optional[Union[str, int]] = None, + is_movie: bool = False, + duration_ms: Optional[Union[str, int]] = None, + mapping: Optional[str] = None, + tvdb_episode_id: Optional[Union[str, int]] = None) -> Tuple[Optional[str], Optional[str]]: + """Build one /media request for the best addressing available. + + The media is named by exactly one id, in the API's own priority order + (tvdb_id -> tmdb_id -> imdb_id): TVDB is the media's identity in v4, and it is + the only id a request can carry the caller's own episode order alongside. A + tmdb_id request is always read as TMDB numbering, which is what made a series + with a non-default display order answer another episode's segments (issue #31). + """ duration_q = '' try: if duration_ms is not None: @@ -282,12 +371,38 @@ def _build_url(tmdb_id: Optional[Union[str, int]], imdb_id: Optional[str], seaso except (TypeError, ValueError): duration_q = '' + order_q = '' + if mapping and str(mapping).strip(): + order_q = '&mapping={}'.format(quote(str(mapping).strip(), safe='')) + + episode_id_q = '' + try: + if tvdb_episode_id is not None: + ep_id = int(str(tvdb_episode_id)) + if ep_id > 0: + episode_id_q = '&tvdb_episode_id={}'.format(ep_id) + except (TypeError, ValueError): + episode_id_q = '' + + if tvdb_id and _valid_numeric_id(tvdb_id): + tid = str(tvdb_id).strip() + if is_movie: + return '{}/media?tvdb_id={}{}'.format(API_BASE, tid, duration_q), 'tvdb' + s, e = _episode_nums(season, episode) + if s is None or e is None or s < 0 or e <= 0: + return None, None + return ( + '{}/media?tvdb_id={}&season={}&episode={}{}{}{}'.format( + API_BASE, tid, s, e, order_q, episode_id_q, duration_q), + 'tvdb', + ) + if tmdb_id and _valid_tmdb(tmdb_id): tid = str(tmdb_id).strip() if is_movie: return '{}/media?tmdb_id={}{}'.format(API_BASE, tid, duration_q), 'tmdb' s, e = _episode_nums(season, episode) - if s is None or e is None or s <= 0 or e <= 0: + if s is None or e is None or s < 0 or e <= 0: return None, None return ( '{}/media?tmdb_id={}&season={}&episode={}{}'.format(API_BASE, tid, s, e, duration_q), @@ -302,35 +417,96 @@ def _build_url(tmdb_id: Optional[Union[str, int]], imdb_id: Optional[str], seaso return '{}/media?imdb_id={}{}'.format(API_BASE, imdb, duration_q), 'imdb' s, e = _episode_nums(season, episode) - if s is None or e is None or s <= 0 or e <= 0: + if s is None or e is None or s < 0 or e <= 0: return None, None return '{}/media?imdb_id={}&season={}&episode={}{}'.format( API_BASE, imdb, s, e, duration_q), 'imdb' -def query_intro(tmdb_id=None, imdb_id=None, season=None, episode=None, is_movie=False, duration_ms: Optional[Union[str, int]] = None): - # returns intro start/end in seconds, or none - if not _is_enabled(): - return None, None +def _media_url_candidates(**kwargs: Any) -> List[Tuple[str, str]]: + """The addressings worth trying for one item, best first. - url, mode = _build_url(tmdb_id, imdb_id, season, episode, is_movie, duration_ms=duration_ms) - if not url: - if tmdb_id or imdb_id: + A 404 is not proof the media is absent: it can be stored under an identity the + request did not carry, which is exactly what the addon hits on streams whose + numbering follows the release rather than TMDB (issue #19). So the request is + retried with the next id, and with the pinned episode id dropped — an episode + id outranks the numbers when the API reads it, so a stale one answers 404 for + an episode that is otherwise present. + """ + candidates: List[Tuple[str, str]] = [] + seen = set() + + attempts = [ + kwargs, + dict(kwargs, tvdb_episode_id=None), + ] + for attempt in attempts: + url, mode = _build_url(**attempt) + if url and url not in seen: + seen.add(url) + candidates.append((url, mode or '')) + + if kwargs.get('tvdb_id'): + for drop_tvdb in (dict(kwargs, tvdb_id=None, tvdb_episode_id=None, mapping=None), + dict(kwargs, tvdb_id=None, tmdb_id=None, tvdb_episode_id=None, mapping=None)): + url, mode = _build_url(**drop_tvdb) + if url and url not in seen: + seen.add(url) + candidates.append((url, mode or '')) + + return candidates + + +def _query_media(tvdb_id: Optional[Union[str, int]] = None, + tmdb_id: Optional[Union[str, int]] = None, + imdb_id: Optional[str] = None, + season: Optional[Union[str, int]] = None, + episode: Optional[Union[str, int]] = None, + is_movie: bool = False, + duration_ms: Optional[Union[str, int]] = None, + mapping: Optional[str] = None, + tvdb_episode_id: Optional[Union[str, int]] = None) -> Tuple[Optional[Dict[str, Any]], Optional[str]]: + """Fetch one media's segments, or (None, None).""" + candidates = _media_url_candidates( + tvdb_id=tvdb_id, tmdb_id=tmdb_id, imdb_id=imdb_id, season=season, + episode=episode, is_movie=is_movie, duration_ms=duration_ms, + mapping=mapping, tvdb_episode_id=tvdb_episode_id) + + if not candidates: + if tvdb_id or tmdb_id or imdb_id: xbmc.log( - '[TheIntroDB] TheIntroDB: need TMDB id, or IMDb tt… id with season/episode for TV', + '[TheIntroDB] TheIntroDB: need a TVDB, TMDB, or IMDb tt… id with season/episode for TV', xbmc.LOGINFO, ) else: - xbmc.log('[TheIntroDB] TheIntroDB: no TMDB or IMDb id', xbmc.LOGINFO) + xbmc.log('[TheIntroDB] TheIntroDB: no TVDB, TMDB or IMDb id', xbmc.LOGINFO) return None, None - xbmc.log('[TheIntroDB] TheIntroDB query ({}): {}'.format(mode, url), xbmc.LOGINFO) + api_key = _get_api_key() + for url, mode in candidates: + if not _wait_rate_limit(): + return None, None + + xbmc.log('[TheIntroDB] TheIntroDB query ({}): {}'.format(mode, url), xbmc.LOGINFO) + status, data = _do_request(url, api_key) + if status == 'ok': + return data, mode + if status != 'not_found': + # Rate limited or unreachable: another id would not be answered either. + return None, None - if not _wait_rate_limit(): + return None, None + + +def query_intro(tmdb_id=None, imdb_id=None, season=None, episode=None, is_movie=False, duration_ms: Optional[Union[str, int]] = None, tvdb_id=None, mapping=None, tvdb_episode_id=None): + # returns intro start/end in seconds, or none + if not _is_enabled(): return None, None - api_key = _get_api_key() - data = _do_request(url, api_key) + data, _mode = _query_media( + tvdb_id=tvdb_id, tmdb_id=tmdb_id, imdb_id=imdb_id, season=season, + episode=episode, is_movie=is_movie, duration_ms=duration_ms, + mapping=mapping, tvdb_episode_id=tvdb_episode_id) if not data: return None, None @@ -349,7 +525,8 @@ def query_intro(tmdb_id=None, imdb_id=None, season=None, episode=None, is_movie= def submit_segment(tmdb_id: Optional[Union[str, int]] = None, imdb_id: Optional[str] = None, season: Optional[Union[str, int]] = None, episode: Optional[Union[str, int]] = None, - is_movie: bool = False, segment: str = 'intro', start_sec: Optional[float] = None, end_sec: Optional[float] = None, video_duration_ms: Optional[Union[str, int]] = None) -> Tuple[bool, str]: + is_movie: bool = False, segment: str = 'intro', start_sec: Optional[float] = None, end_sec: Optional[float] = None, video_duration_ms: Optional[Union[str, int]] = None, + tvdb_id: Optional[Union[str, int]] = None, tvdb_episode_id: Optional[Union[str, int]] = None) -> Tuple[bool, str]: """Submit a segment timestamp to TheIntroDB. Returns (success, message) tuple. @@ -358,10 +535,11 @@ def submit_segment(tmdb_id: Optional[Union[str, int]] = None, imdb_id: Optional[ if not api_key: return False, 'API key required for submissions. Set it in addon settings.' + has_tvdb = _valid_numeric_id(tvdb_id) if not tmdb_id or not _valid_tmdb(tmdb_id): - # fall back to imdb_id if we have one — the API can resolve it - if not _normalize_imdb(imdb_id): - return False, 'Need a TMDB or IMDb ID to submit.' + # fall back to tvdb_id or imdb_id if we have one — the API can resolve it + if not has_tvdb and not _normalize_imdb(imdb_id): + return False, 'Need a TVDB, TMDB or IMDb ID to submit.' if not _wait_rate_limit(): return False, 'Rate limited. Try again later.' @@ -370,6 +548,14 @@ def submit_segment(tmdb_id: Optional[Union[str, int]] = None, imdb_id: Optional[ 'segment': segment, } + # TVDB is the media's identity, so it is the preferred addressing: a tmdb_id + # submission is validated against TMDB numbering, which rejects an episode + # TMDB does not list. + if has_tvdb: + payload['tvdb_id'] = int(str(tvdb_id).strip()) + if not is_movie and _valid_numeric_id(tvdb_episode_id) and int(str(tvdb_episode_id)) != payload['tvdb_id']: + payload['tvdb_episode_id'] = int(str(tvdb_episode_id)) + if tmdb_id and _valid_tmdb(tmdb_id): payload['tmdb_id'] = int(str(tmdb_id).strip()) @@ -458,29 +644,15 @@ def submit_segment(tmdb_id: Optional[Union[str, int]] = None, imdb_id: Optional[ return False, str(e) -def query_all_segments(tmdb_id: Optional[Union[str, int]] = None, imdb_id: Optional[str] = None, season: Optional[Union[str, int]] = None, episode: Optional[Union[str, int]] = None, is_movie: bool = False, duration_ms: Optional[Union[str, int]] = None) -> Dict[str, Any]: +def query_all_segments(tmdb_id: Optional[Union[str, int]] = None, imdb_id: Optional[str] = None, season: Optional[Union[str, int]] = None, episode: Optional[Union[str, int]] = None, is_movie: bool = False, duration_ms: Optional[Union[str, int]] = None, tvdb_id: Optional[Union[str, int]] = None, mapping: Optional[str] = None, tvdb_episode_id: Optional[Union[str, int]] = None) -> Dict[str, Any]: # returns dict with all segment types and their segments if not _is_enabled(): return {} - url, mode = _build_url(tmdb_id, imdb_id, season, episode, is_movie, duration_ms=duration_ms) - if not url: - if tmdb_id or imdb_id: - xbmc.log( - '[TheIntroDB] TheIntroDB: need TMDB id, or IMDb tt… id with season/episode for TV', - xbmc.LOGINFO, - ) - else: - xbmc.log('[TheIntroDB] TheIntroDB: no TMDB or IMDb id', xbmc.LOGINFO) - return {} - - xbmc.log('[TheIntroDB] TheIntroDB query all segments ({}): {}'.format(mode, url), xbmc.LOGINFO) - - if not _wait_rate_limit(): - return {} - - api_key = _get_api_key() - data = _do_request(url, api_key) + data, _mode = _query_media( + tvdb_id=tvdb_id, tmdb_id=tmdb_id, imdb_id=imdb_id, season=season, + episode=episode, is_movie=is_movie, duration_ms=duration_ms, + mapping=mapping, tvdb_episode_id=tvdb_episode_id) if not data: return {} diff --git a/plugin.video.tidb/player.py b/plugin.video.tidb/player.py index 22b6462..90e9db4 100644 --- a/plugin.video.tidb/player.py +++ b/plugin.video.tidb/player.py @@ -123,6 +123,8 @@ def get_media_ids(self) -> Dict[str, Any]: ids: Dict[str, Any] = { 'imdb_id': None, 'tmdb_id': None, + 'tvdb_id': None, + 'tvdb_episode_id': None, 'season': None, 'episode': None, 'is_movie': False, @@ -133,6 +135,12 @@ def get_media_ids(self) -> Dict[str, Any]: self._ids_from_infotag(ids) ids['duration_ms'] = self._get_video_duration_ms() + # Which id lands on an episode depends on the scraper: some store the + # episode record, others repeat the series id. Only a distinct one is a + # useful pin — sent as tvdb_id the API would look for a series by it. + if ids.get('tvdb_episode_id') and ids['tvdb_episode_id'] == ids.get('tvdb_id'): + ids['tvdb_episode_id'] = None + xbmc.log('[TheIntroDB] Extracted media IDs: {}'.format(ids), xbmc.LOGINFO) return ids @@ -316,6 +324,15 @@ def _apply_uniqueid_dict(self, unique: Any, ids: Dict[str, Any]) -> None: if imdb_val and str(imdb_val).startswith('tt'): ids['imdb_id'] = str(imdb_val) + if not ids.get('tvdb_id'): + tvdb_val = unique.get('tvdb') or unique.get('thetvdb') + if tvdb_val: + try: + if int(tvdb_val) > 0: + ids['tvdb_id'] = str(tvdb_val) + except (ValueError, TypeError): + pass + if not ids.get('tmdb_id'): tmdb_show = unique.get('tmdbshow') or unique.get('tmdb_show') if tmdb_show: @@ -343,6 +360,26 @@ def _apply_episode_item_uniqueid_dict(self, unique: Any, ids: Dict[str, Any]) -> if imdb_val and str(imdb_val).startswith('tt'): ids['imdb_id'] = str(imdb_val) + # On an episode, the TVDB scraper's `tvdb` uniqueid is the episode record + # (the series id arrives with the show's own uniqueid below). + if not ids.get('tvdb_episode_id'): + tvdb_val = unique.get('tvdb') + if tvdb_val: + try: + if int(tvdb_val) > 0: + ids['tvdb_episode_id'] = str(tvdb_val) + except (ValueError, TypeError): + pass + + if not ids.get('tvdb_id'): + tvdb_show = unique.get('tvdbshow') or unique.get('tvdb_show') + if tvdb_show: + try: + if int(tvdb_show) > 0: + ids['tvdb_id'] = str(tvdb_show) + except (ValueError, TypeError): + pass + def _tvshow_ids_from_library(self, tvshowid: Any, ids: Dict[str, Any], force_tmdb: bool = False) -> None: try: tvshowid_int = int(tvshowid) @@ -368,6 +405,15 @@ def _tvshow_ids_from_library(self, tvshowid: Any, ids: Dict[str, Any], force_tmd if imdb_unique and str(imdb_unique).startswith('tt'): ids['imdb_id'] = str(imdb_unique) + if not ids.get('tvdb_id'): + tvdb_unique = (unique or {}).get('tvdb') if isinstance(unique, dict) else None + if tvdb_unique: + try: + if int(tvdb_unique) > 0: + ids['tvdb_id'] = str(tvdb_unique) + except (ValueError, TypeError): + pass + imdbnumber = details.get('imdbnumber') if imdbnumber and str(imdbnumber).startswith('tt') and not ids.get('imdb_id'): ids['imdb_id'] = str(imdbnumber) @@ -406,6 +452,15 @@ def _tvshow_ids_by_title(self, title: Optional[str], ids: Dict[str, Any], force_ if imdb_unique and str(imdb_unique).startswith('tt'): ids['imdb_id'] = str(imdb_unique) + if not ids.get('tvdb_id'): + tvdb_unique = (unique or {}).get('tvdb') if isinstance(unique, dict) else None + if tvdb_unique: + try: + if int(tvdb_unique) > 0: + ids['tvdb_id'] = str(tvdb_unique) + except (ValueError, TypeError): + pass + imdbnumber = pick.get('imdbnumber') if imdbnumber and str(imdbnumber).startswith('tt') and not ids.get('imdb_id'): ids['imdb_id'] = str(imdbnumber) @@ -465,6 +520,44 @@ def _ids_from_infotag(self, ids: Dict[str, Any]) -> None: except Exception: pass + # TVDB is the API's identity, so the id is worth collecting even when a + # TMDB one exists. On an episode tag `tvdb` is the episode record and the + # series id lives on the show (tvdb_show / tvdbshow) — the two are told + # apart by the media type, and get_media_ids drops the episode id when a + # scraper repeated the series id on it. + try: + media_type = tag.getMediaType() + except Exception: + media_type = None + + try: + tvdb = tag.getUniqueID('tvdb') + if tvdb and not str(tvdb).startswith('tt'): + val = str(tvdb) + if int(val) > 0: + if media_type == 'movie': + if not ids.get('tvdb_id'): + ids['tvdb_id'] = val + elif not ids.get('tvdb_episode_id'): + ids['tvdb_episode_id'] = val + except Exception: + pass + + if not ids.get('tvdb_id'): + for key in ('tvdb_show', 'tvdbshow'): + try: + tvdb_show = tag.getUniqueID(key) + except Exception: + tvdb_show = None + if not tvdb_show: + continue + try: + if int(str(tvdb_show)) > 0: + ids['tvdb_id'] = str(tvdb_show) + break + except (ValueError, TypeError): + continue + if not ids['tmdb_id']: try: for key in ('tmdbshow', 'tmdb_show', 'themoviedb', 'tmdb'): diff --git a/plugin.video.tidb/service.py b/plugin.video.tidb/service.py index 6dc0d4a..fc3ce58 100644 --- a/plugin.video.tidb/service.py +++ b/plugin.video.tidb/service.py @@ -380,6 +380,8 @@ def _mark_end_and_submit(session: PlaybackSession, player: TIDBPlayer, monitor: success, msg = introdb.submit_segment( tmdb_id=media_ids.get('tmdb_id'), imdb_id=media_ids.get('imdb_id'), + tvdb_id=media_ids.get('tvdb_id'), + tvdb_episode_id=media_ids.get('tvdb_episode_id'), season=media_ids.get('season'), episode=media_ids.get('episode'), is_movie=media_ids.get('is_movie', False), @@ -447,11 +449,13 @@ def _run_service() -> None: # ── Fetch media IDs (cached) ── if session.media_ids is None: session.media_ids = player.get_media_ids() - xbmc.log('[TheIntroDB] Media IDs: tmdb={} imdb={} S{}E{} movie={}'.format( + xbmc.log('[TheIntroDB] Media IDs: tvdb={} tvdb_ep={} tmdb={} imdb={} S{}E{} movie={}'.format( + session.media_ids.get('tvdb_id'), session.media_ids.get('tvdb_episode_id'), session.media_ids.get('tmdb_id'), session.media_ids.get('imdb_id'), session.media_ids.get('season'), session.media_ids.get('episode'), session.media_ids.get('is_movie', False)), xbmc.LOGINFO) media_ids = session.media_ids + tvdb = media_ids.get('tvdb_id') tmdb = media_ids.get('tmdb_id') imdb = media_ids.get('imdb_id') m_season = media_ids.get('season') @@ -466,9 +470,11 @@ def _run_service() -> None: # ── Fetch segments (cached) ── all_segments = {} - if introdb_on and (tmdb or imdb): + if introdb_on and (tvdb or tmdb or imdb): if session.all_segments is None: session.all_segments = introdb.query_all_segments( + tvdb_id=tvdb, + tvdb_episode_id=media_ids.get('tvdb_episode_id'), tmdb_id=tmdb, imdb_id=imdb, season=m_season, episode=m_episode, is_movie=m_movie, duration_ms=media_ids.get('duration_ms'), diff --git a/tests/test_introdb_v4.py b/tests/test_introdb_v4.py new file mode 100644 index 0000000..cea4256 --- /dev/null +++ b/tests/test_introdb_v4.py @@ -0,0 +1,212 @@ +# SPDX-License-Identifier: GPL-2.0-or-later +# Copyright (C) 2026 TheIntroDB +# +"""Unit tests for the API client's v4 request shape. + +Kodi's Python modules don't exist off-device, so they are stubbed before the +addon module is imported. +""" +import json +import os +import sys +import types +import unittest +from email.message import Message + +sys.path.insert(0, os.path.join(os.path.dirname(os.path.dirname(os.path.abspath(__file__))), 'plugin.video.tidb')) + +_SETTINGS = {} + + +class _Addon: + def getAddonInfo(self, key): + return 'plugin.video.tidb' if key == 'id' else '' + + def getSetting(self, key): + return _SETTINGS.get(key, '') + + +xbmc_stub = types.ModuleType('xbmc') +xbmc_stub.LOGINFO = 0 +xbmc_stub.LOGWARNING = 1 +xbmc_stub.LOGERROR = 2 +xbmc_stub.log = lambda *args, **kwargs: None +xbmc_stub.Monitor = object +xbmcaddon_stub = types.ModuleType('xbmcaddon') +xbmcaddon_stub.Addon = _Addon +sys.modules.setdefault('xbmc', xbmc_stub) +sys.modules.setdefault('xbmcaddon', xbmcaddon_stub) + +import introdb # noqa: E402 + +_SETTINGS['introdb_enabled'] = 'true' + + +def _headers(**values): + message = Message() + for key, value in values.items(): + message[key.replace('_', '-')] = str(value) + return message + + +class _HttpError(introdb.HTTPError): + def __init__(self, code, headers, body=''): + super().__init__('http://example.invalid', code, 'error', headers, None) + self._body = body + + def read(self): + return self._body.encode('utf-8') + + +class ApiVersionTests(unittest.TestCase): + def test_client_speaks_v4(self): + self.assertEqual('v4', introdb.API_VERSION) + self.assertEqual('https://api.theintrodb.org/v4', introdb.API_BASE) + + +class UrlBuildingTests(unittest.TestCase): + def test_tvdb_is_preferred_and_carries_order_and_episode_id(self): + url, mode = introdb._build_url( + tvdb_id=73871, tmdb_id=615, imdb_id='tt0149460', + season=7, episode=1, is_movie=False, + duration_ms=1297000, mapping='alternate', tvdb_episode_id=1051911) + self.assertEqual('tvdb', mode) + self.assertEqual( + 'https://api.theintrodb.org/v4/media?tvdb_id=73871&season=7&episode=1' + '&mapping=alternate&tvdb_episode_id=1051911&duration_ms=1297000', + url) + + def test_tmdb_request_drops_order_and_episode_id(self): + # A tmdb_id request is always read as TMDB numbering, so an order beside it + # would be a lie rather than a hint. + url, mode = introdb._build_url( + tmdb_id=615, season=7, episode=1, mapping='alternate', tvdb_episode_id=1051911) + self.assertEqual('tmdb', mode) + self.assertEqual('https://api.theintrodb.org/v4/media?tmdb_id=615&season=7&episode=1', url) + + def test_movie_urls_and_imdb_fallback(self): + self.assertEqual( + ('https://api.theintrodb.org/v4/media?tvdb_id=169', 'tvdb'), + introdb._build_url(tvdb_id=169, is_movie=True)) + self.assertEqual( + ('https://api.theintrodb.org/v4/media?imdb_id=tt0111161', 'imdb'), + introdb._build_url(imdb_id='tt0111161', is_movie=True)) + + def test_tv_request_needs_a_usable_episode_number(self): + self.assertEqual((None, None), introdb._build_url(tmdb_id=615, season=0, episode=0)) + + def test_candidates_fall_back_to_the_next_identity(self): + candidates = introdb._media_url_candidates( + tvdb_id=73871, tmdb_id=615, imdb_id='tt0149460', season=7, episode=1, + is_movie=False, duration_ms=None, mapping='alternate', tvdb_episode_id=1051911) + self.assertEqual([ + 'https://api.theintrodb.org/v4/media?tvdb_id=73871&season=7&episode=1&mapping=alternate&tvdb_episode_id=1051911', + 'https://api.theintrodb.org/v4/media?tvdb_id=73871&season=7&episode=1&mapping=alternate', + 'https://api.theintrodb.org/v4/media?tmdb_id=615&season=7&episode=1', + 'https://api.theintrodb.org/v4/media?imdb_id=tt0149460&season=7&episode=1', + ], [url for url, _mode in candidates]) + + +class QueryFallbackTests(unittest.TestCase): + def setUp(self): + self.calls = [] + self._original_request = introdb._do_request + self._original_wait = introdb._wait_rate_limit + introdb._wait_rate_limit = lambda: True + + def tearDown(self): + introdb._do_request = self._original_request + introdb._wait_rate_limit = self._original_wait + + def _responder(self, statuses): + def fake(url, api_key): + self.calls.append(url) + status = statuses.pop(0) + if status == 'ok': + return 'ok', {'intro': [{'start_ms': None, 'end_ms': 29344}]} + return status, None + introdb._do_request = fake + + def test_404_moves_on_to_the_next_identity(self): + self._responder(['not_found', 'ok']) + data, mode = introdb._query_media(tvdb_id=73871, tmdb_id=615, season=7, episode=1) + # The tvdb addressing found nothing, so the same episode was asked for by + # tmdb_id — the id a stream's metadata usually carries. + self.assertEqual('tmdb', mode) + self.assertIsNotNone(data) + self.assertEqual(2, len(self.calls)) + self.assertIn('tvdb_id=73871', self.calls[0]) + self.assertIn('tmdb_id=615', self.calls[1]) + + def test_rate_limit_stops_the_fallback_chain(self): + self._responder(['rate_limited']) + data, mode = introdb._query_media(tvdb_id=73871, tmdb_id=615, season=7, episode=1) + self.assertIsNone(data) + self.assertIsNone(mode) + self.assertEqual(1, len(self.calls)) + + def test_every_identity_missing_returns_nothing(self): + self._responder(['not_found', 'not_found', 'not_found']) + data, mode = introdb._query_media(tvdb_id=73871, tmdb_id=615, imdb_id='tt0149460', season=7, episode=1) + self.assertIsNone(data) + self.assertIsNone(mode) + self.assertEqual(3, len(self.calls)) + + +class RetryAfterTests(unittest.TestCase): + def test_retry_after_header_is_preferred(self): + error = _HttpError(429, _headers(Retry_After=10, X_RateLimit_Reset=999999)) + self.assertEqual(10, introdb._retry_after_seconds(error, '')) + + def test_rate_limit_reset_is_clamped_to_the_burst_window(self): + error = _HttpError(429, _headers(X_RateLimit_Reset=999999)) + self.assertEqual(introdb.RATE_RESET_CLAMP, introdb._retry_after_seconds(error, '')) + + def test_usage_limit_reset_is_trusted_up_to_the_daily_ceiling(self): + body = json.dumps({'error': 'Usage limit exceeded', 'code': 'usage_limit_exceeded'}) + error = _HttpError(429, _headers(X_UsageLimit_Reset=49000), body) + self.assertEqual(49000, introdb._retry_after_seconds(error, body)) + + error = _HttpError(429, _headers(X_UsageLimit_Reset=999999), body) + self.assertEqual(introdb.USAGE_RESET_CLAMP, introdb._retry_after_seconds(error, body)) + + def test_specific_media_usage_limit_is_recognized_by_its_code(self): + body = json.dumps({'error': 'Usage limit exceeded', 'code': 'specific_media_usage_limit_exceeded'}) + error = _HttpError(429, _headers(Retry_After=7200), body) + self.assertEqual(7200, introdb._retry_after_seconds(error, body)) + + +class ApiKeyProbeTests(unittest.TestCase): + def setUp(self): + self._original = introdb.urlopen + self.urls = [] + + def tearDown(self): + introdb.urlopen = self._original + _SETTINGS.pop('introdb_api_key', None) + + def test_probe_targets_user_stats(self): + def fake(request, timeout=None): + self.urls.append(request.full_url) + raise _HttpError(401, _headers()) + + introdb.urlopen = fake + _SETTINGS['introdb_api_key'] = 'theintrodb:user_1:secret' + valid, message = introdb.test_api_key() + self.assertFalse(valid) + self.assertIn('401', message) + self.assertEqual(['https://api.theintrodb.org/v4/user/stats'], self.urls) + + def test_probe_reports_an_unexpected_status_instead_of_passing(self): + def fake(request, timeout=None): + raise _HttpError(404, _headers()) + + introdb.urlopen = fake + _SETTINGS['introdb_api_key'] = 'theintrodb:user_1:secret' + valid, message = introdb.test_api_key() + self.assertFalse(valid) + self.assertIn('404', message) + + +if __name__ == '__main__': + unittest.main() From bd75ca6855d6046cc15a1d78d83a8dbe0eceee8a Mon Sep 17 00:00:00 2001 From: Pas <74743263+Pasithea0@users.noreply.github.com> Date: Sat, 17 Oct 2026 14:22:25 -0600 Subject: [PATCH 2/3] fix: bound the submit path's 429 with the same retry logic The submission handler parsed the rate-limit headers inline: it took whichever of X-UsageLimit-Reset / X-RateLimit-Reset / Retry-After came first, with no ceiling, no floor and no check of what the value meant. An epoch-stated or oversized reset parked the client for an arbitrarily long time, and since the lookup path shares that deadline, one bad submit response also silenced segment lookups. Route it through _retry_after_seconds, the parser the lookup path already uses, and initialize the error body so the helper always gets one. Adds tests for the submit 429: the daily budget's reset is trusted to its own ceiling, and a burst reset is clamped and can never land in the past. --- plugin.video.tidb/introdb.py | 13 ++++-------- tests/test_introdb_v4.py | 39 ++++++++++++++++++++++++++++++++++++ 2 files changed, 43 insertions(+), 9 deletions(-) diff --git a/plugin.video.tidb/introdb.py b/plugin.video.tidb/introdb.py index 9417a43..8dbf269 100644 --- a/plugin.video.tidb/introdb.py +++ b/plugin.video.tidb/introdb.py @@ -616,6 +616,7 @@ def submit_segment(tmdb_id: Optional[Union[str, int]] = None, imdb_id: Optional[ return True, 'Submitted! Status: {}'.format(status) return True, 'Submitted successfully.' except HTTPError as e: + err_body = '' try: err_body = e.read().decode('utf-8') err_data = json.loads(err_body) @@ -623,15 +624,9 @@ def submit_segment(tmdb_id: Optional[Union[str, int]] = None, imdb_id: Optional[ except Exception: err_msg = 'HTTP {}'.format(e.code) if e.code == 429: - retry = 300 - for header in ('X-UsageLimit-Reset', 'X-RateLimit-Reset', 'Retry-After'): - val = e.headers.get(header) - if val: - try: - retry = int(val) - except ValueError: - pass - break + # Same bounded parsing as the lookup path: this budget is shared, so + # one oversized or epoch-stated reset must not silence lookups too. + retry = _retry_after_seconds(e, err_body) with _rate_limit_lock: _rate_limit_until = time.time() + retry xbmc.log('[TheIntroDB] Submit failed: {}'.format(err_msg), xbmc.LOGWARNING) diff --git a/tests/test_introdb_v4.py b/tests/test_introdb_v4.py index cea4256..a24e614 100644 --- a/tests/test_introdb_v4.py +++ b/tests/test_introdb_v4.py @@ -9,6 +9,7 @@ import json import os import sys +import time import types import unittest from email.message import Message @@ -208,5 +209,43 @@ def fake(request, timeout=None): self.assertIn('404', message) +class Submit429Tests(unittest.TestCase): + """A submission's 429 shares the lookup budget, so it must be bounded too.""" + + def setUp(self): + self._original_urlopen = introdb.urlopen + self._original_wait = introdb._wait_rate_limit + introdb._wait_rate_limit = lambda: True + introdb._rate_limit_until = 0.0 + _SETTINGS['introdb_api_key'] = 'theintrodb:user_1:secret' + + def tearDown(self): + introdb.urlopen = self._original_urlopen + introdb._wait_rate_limit = self._original_wait + introdb._rate_limit_until = 0.0 + _SETTINGS.pop('introdb_api_key', None) + + def _submit(self, headers, body=''): + def fake(request, timeout=None): + raise _HttpError(429, headers, body) + + introdb.urlopen = fake + ok, _message = introdb.submit_segment( + tvdb_id=169, is_movie=True, segment='intro', start_sec=30, end_sec=90) + self.assertFalse(ok) + return introdb._rate_limit_until - time.time() + + def test_daily_budget_reset_is_trusted(self): + body = json.dumps({'error': 'Usage limit exceeded', 'code': 'usage_limit_exceeded'}) + remaining = self._submit(_headers(X_UsageLimit_Reset=49000, Retry_After=49000), body) + self.assertGreater(remaining, 48000) + self.assertLess(remaining, 49001) + + def test_burst_reset_is_clamped_and_never_lands_in_the_past(self): + remaining = self._submit(_headers(X_RateLimit_Reset=999999)) + self.assertGreater(remaining, 0) + self.assertLessEqual(remaining, introdb.RATE_RESET_CLAMP) + + if __name__ == '__main__': unittest.main() From 736e8079864052361358e47708fd29cacd17d55b Mon Sep 17 00:00:00 2001 From: Pas <74743263+Pasithea0@users.noreply.github.com> Date: Sat, 17 Oct 2026 14:28:14 -0600 Subject: [PATCH 3/3] docs: stop claiming the episode pin chooses the episode Measured against production on 2026-10-17: /v4/media answers from season/episode, not from tvdb_episode_id, and does not validate it at all. With Futurama the episode record 4319164 (default S07E01, 'The Bots and the Bees') was pinned alongside season=6&episode=1 and the response carried S06E1's timestamps ('Rebirth', 29344 ms); pinning 1051911 (default S06E1) alongside season=7&episode=1 returned S07E1's 14600 ms. A junk id is accepted with a 200, a valid id does not rescue numbers that resolve to nothing, and a pin with no season/episode is a 400, so it is never the addressing. The README described the pin as making the numbering irrelevant. It does not, and a client that trusts it can show another episode's intro, so say what the API actually does and what it costs a library scraped in a non-default order. The request still carries the pin, which is correct once the documented precedence is honoured. --- README.md | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/README.md b/README.md index d089068..65f4ac9 100644 --- a/README.md +++ b/README.md @@ -41,11 +41,11 @@ Kodi service add-on that gets intro, recap, credits, and preview segments from * **TVDB is recommended.** TVDB is TheIntroDB's identity, so it is what the add-on asks with first; **TMDB** is next, then **IMDb** `tt...`. When the item exposes several, the add-on uses the best one and falls back to the others if the first finds nothing. -Alongside the id, an episode's own **TVDB episode id** is sent when Kodi has one, which pins the exact episode instead of trusting the numbering. +Alongside the id, an episode's own **TVDB episode id** is sent when Kodi has one. It is the add-on's way of saying which exact episode record it means; note that the API currently answers `/media` from the season/episode numbers rather than that id (measured 2026-10-17: a pinned id naming a different episode than the numbers returns the *numbers'* episode), so the numbering is still what decides the answer. The id is sent for when that precedence is honoured. For TV episodes, the playing item also needs valid **season** and **episode** numbers. If your source add-on or library does not expose provider IDs, TheIntroDB cannot match the item. -Known limitation: for a series whose episode numbers follow a release order rather than TVDB's default (streaming rips, absolute-numbered anime, some specials), a lookup addressed only by **TMDB** can miss content that is stored under another numbering. Kodi items carrying a TVDB id resolve exactly; for the others, passing season/episode as the release numbers them is all the add-on can do. +Known limitation: for a series whose episode numbers follow a release order rather than TVDB's default (streaming rips, absolute-numbered anime, some specials), a lookup is answered from the numbers as TVDB's *default* order, which can be a different episode than the one you are watching. Kodi does not expose which order its numbers came from, so the add-on cannot pass an order to convert; if your library is scraped in a non-default order, prefer a source that carries a TVDB id and check the result against the episode you are watching. ## How It Works