http-cache-rules

Client-independent HTTP caching rules for Python: RFC 9111 and the RFC 5861 extensions, with no runtime dependencies.

from http_cache_rules import Action, CachePolicy, Headers, Request, Response

request = Request("GET", "https://example.test/data", Headers({"Accept-Language": "en"}))
response = Response(200, Headers([("Cache-Control", "max-age=60"), ("ETag", '"v1"')]))

policy = CachePolicy(request, response, shared=True)
if policy.storable():
    action = policy.evaluate(request)        # Action.SERVE while fresh
    headers = policy.response_headers()      # stored headers plus a computed Age
    conditional = policy.revalidation_headers()  # If-None-Match / If-Modified-Since

Why

Python HTTP caches embed their policy in a client: CacheControl wraps requests, requests-cache wraps requests, hishel wraps httpx. Code that needs the decisions without the client (a proxy, an async framework, a custom fetcher) has to reimplement them. hishel's SyncCacheProxy is client-independent, but it brings its storage layer, and its decisions are the ones measured below.

Measured: the http-tests/cache-tests suite at commit d644cf4, each library wrapped in a stdlib reverse proxy (bench/suite.sh for this package). "Decided" means passed plus failed; tests that depend on a failed test, or that need a browser, are not counted.

Library	Required pass / fail	% of decided required	Optional pass / fail
http-cache-rules (shared)	147 / 0	100.0	84 / 11
hishel 1.4.0 (SyncCacheProxy, no httpx)	108 / 18	85.7	65 / 23
hishel 1.4.0 (httpx client)	106 / 18	85.5	65 / 23
CacheControl 0.14.4	77 / 21	78.6	33 / 49
requests-cache 1.3.3	81 / 29	73.6	44 / 38

The suite excludes its cdn-cache-control and interim groups from these counts. The incumbents ran in their default configuration through the same harness style (requests-cache with expire_after=0, otherwise it caches everything forever). Their counts move by one or two tests between runs. The suite's own README notes that passing it proves little, and it says nothing about behavior it does not test.

The failures common to the incumbents that this package handles: invalid Expires accepted as valid, no computed Age on reuse, Vary values like *, * or , * treated as a normal field list, hop-by-hop and Connection-listed headers stored, and invalidation by PUT/DELETE/unknown methods.

Installation

pip install http-cache-rules

Python 3.11 or newer. No dependencies.

Usage

Deciding how to answer a request

evaluate(request, now) returns an Action: SERVE, SERVE_STALE, SERVE_STALE_THEN_REVALIDATE, REVALIDATE or MISS. It applies request Cache-Control (no-cache, max-age, min-fresh, max-stale), Pragma: no-cache, response no-cache, must-revalidate, proxy-revalidate, s-maxage and stale-while-revalidate.

Revalidating

Add policy.revalidation_headers() to the forwarded request. On a 304, policy.update(response) returns the merged policy. On any other response, build a new CachePolicy.

Serving errors

policy.can_serve_stale(now, error=True) answers whether a stale response may cover an origin failure, honoring stale-if-error and the prohibitions above.

Invalidation

invalidated_urls(request, response) lists the URLs to drop after a successful unsafe request. policy.invalidates(request, response) checks one stored policy.

Persistence

policy.to_dict() returns JSON-safe data and CachePolicy.from_dict() restores it. The policy holds headers only. Bodies belong to the caller.

Client adapters

examples/adapters.py builds policies from httpx and requests response objects without importing either library. It is an example, not part of the package.

Reference

CachePolicy(request, response, *, shared=True, request_time=None, response_time=None, heuristic_fraction=0.1, heuristic_max=None, cacheable_methods=("GET", "HEAD"))

Option	Type	Default	Description
shared	bool	True	Shared cache (honors s-maxage, private, proxy-revalidate, Authorization rules) or private cache.
request_time	float | None	response_time	POSIX time the request was sent, for response-delay correction.
response_time	float | None	now	POSIX time the response arrived.
heuristic_fraction	float	0.1	Fraction of Date - Last-Modified used as heuristic freshness. 0 disables.
heuristic_max	float | None	None	Upper bound for heuristic freshness in seconds.
cacheable_methods	iterable of str	GET, HEAD	Methods whose responses may be stored.

Method	Returns	Description
storable()	bool	RFC 9111 section 3 storage rules.
satisfies(request)	bool	URL, method and every Vary rule. A * anywhere in Vary never matches.
freshness_lifetime()	float	s-maxage (shared), max-age, Expires - Date, then heuristic. Invalid information gives 0.
age(now)	float	Current age with Age, Date and response delay.
is_fresh(now)	bool	Age below lifetime. Ignores no-cache and request directives.
can_serve_stale(now, error=False, request=None)	bool	Stale reuse allowed by stale-while-revalidate, request max-stale, or (with error) stale-if-error.
evaluate(request, now)	Action	Combines the above for one request.
response_headers(now, revalidated=False)	Headers	Stored headers with a computed Age. Date is not rewritten. Qualified no-cache and (shared) private fields are removed.
revalidation_headers()	Headers	If-None-Match and If-Modified-Since.
validates(response)	bool	Whether the response's validators match the stored ones.
update(response, *, request_time, response_time)	CachePolicy	Section 3.2 merge. Raises ValueError when validates is false.
invalidates(unsafe_request, response)	bool	Section 4.4 for this policy's URL.
conditional_matches(request)	bool	Whether a client's If-None-Match or If-Modified-Since allows a 304.
to_dict() / from_dict()		Persistence.

Other exports: Request, Response (dataclasses with Headers), Headers (case-insensitive, repeated lines kept separate), CacheControl (strict parser), parse_http_date, format_http_date, invalidated_urls.

Parsing rules: directive names are case-insensitive; quoted values are accepted; a member with a space around = is ignored; max-age must be 1*DIGIT (larger than 2^31 is capped) and anything else, or two disagreeing values, counts as stale. Dates must be IMF-fixdate, RFC 850 or asctime with GMT; anything else is invalid, and an invalid Expires means expired. Age uses the first list member of the first line and is ignored when not 1*DIGIT.

How it works

The policy stores the response headers (minus hop-by-hop and Connection-listed fields), the status, the timestamps, and only the request headers that matter later: those named by Vary, a marker for Authorization and no-store. Every decision is computed from those on demand, from an explicit now, so the object is deterministic and testable.

Limitations

• Headers only. No body, no storage, no key management, no I/O.
• 206 and range handling are not implemented: partial responses are not storable and there is no range combining.
• POST responses are storable only by listing POST in cacheable_methods; the Content-Location reuse rule is not implemented.
• The weekday in an HTTP-date is checked for shape, not for agreement with the date.
• Vary normalization covers whitespace, list combining, and case and order for Accept-Language and Accept-Encoding. It does not select by q-value.
• If-Modified-Since with no Last-Modified compares against Date.
• immutable (RFC 8246) is not interpreted. Warning headers are not generated.
• The suite's cdn-cache-control and interim groups are not scored.

Suite results that are not passes are listed in the Compatibility section.

Compatibility

Run on Python 3.14.7 (macOS) only: pytest, mypy --strict, and the suite. Python 3.11 to 3.13 are configured in CI and not run here. The suite needs Node and git; it was run with the Node on this machine.

Suite (bench/suite.sh, shared mode, three runs): required 147 passed, 0 failed, 2 not decided (dependency on a failed partial test), 3 not decided (browser only); optional 84 passed, 11 failed. All three runs gave identical counts. Not passing optional tests, shared mode:

• method-POST: needs the Content-Location rule above.
• vary-normalise-lang-select: selects a stored variant by q value; not implemented.
• conditional-lm-fresh-no-lm: expects 304 for an If-Modified-Since older than the response's Date when there is no Last-Modified; the rules in RFC 9110 give 200, so this package answers 200.
• partial-* (7 tests): range support is not implemented.

Not decided: cc-resp-immutable-* (browser only), and the required partial-use-headers and partial-use-stored-headers (they depend on the optional partial tests, which fail here).

In private mode (bench/suite.sh --private) the private-only tests pass and the shared-only tests (freshness-s-maxage-shared, cc-resp-private-shared, stale-close-proxy-revalidate, stale-close-s-maxage=2, other-authorization) fail by design: 139 required passed and those 5 failed, 5 not decided; optional 79 passed and 11 failed.

Development

python3 -m venv .venv && .venv/bin/pip install -e '.[dev]'
.venv/bin/python -m pytest -q
.venv/bin/python -m mypy --strict
bench/suite.sh            # fetches cache-tests (BSD-3-Clause) into .cache-tests/ at a pinned commit
bench/suite.sh --private

bench/suite.sh clones the suite into the gitignored .cache-tests/, checks out commit d644cf4bf487763646aac19d2c8b846daa0f604d, verifies HEAD, and runs npm install there (the suite ships no lockfile). Nothing from the suite is committed. bench/proxy.py is the reverse-proxy cache; all caching decisions in it come from this package.

License

MIT. See LICENSE.