Skip to content

Commit a287ee2

Browse files
fix(signing)!: stop allow_private from admitting IANA special-use ranges (#1377)
fix(signing)!: stop `allow_private` from admitting IANA special-use ranges Closes #1357 `validate_resolved_ip` applied its whole blocked set behind one boolean, so `allow_private=True` — documented for dialling a test's own loopback origin — also admitted 23 named IANA ranges that are not private address space, among them the RFC 6598 carrier-grade NAT and RFC 7526/7535/7450/7534/7343 ranges PR #974 added, plus the unallocated IPv6 space `is_reserved` covers. A caller that enabled private destinations lost the special-use blocklist with them, on the paths where the dialled URL is publisher- or buyer-supplied. ## What changed The blocked set splits at the reachability line, and each half gets its own gate: * `allow_private` admits **private destinations** — RFC 1918, RFC 4193 unique-local, loopback. These are the ranges an operator controls and the ones the flag's docstring describes. * `allow_special_use` admits **special-use ranges** — link-local, multicast, the unspecified address, RFC 5737/3849 documentation space, RFC 2544 benchmarking, RFC 6890 protocol assignments, `240.0.0.0/4`, 6to4, Teredo, NAT64, and the `_EXTRA_BLOCKED_NETWORKS` anycast and identifier ranges. None is a destination an operator deploys a service to. * `BLOCKED_METADATA_IPS` stays blocked under both, as before. `_PRIVATE_DESTINATION_NETWORKS` holds the admitted set, next to `_EXTRA_BLOCKED_NETWORKS` and with the same kind of per-range justification. `_is_blocked_by_base_policy` spells the pre-split union once, so the two gates partition that union rather than each restating part of it. `resolve_and_validate_host`, `validate_jwks_uri`, `build_ip_pinned_transport`, `build_async_ip_pinned_transport` and `abuild_ip_pinned_transport` take the new keyword and forward it. The higher-level agent-resolution and brand.json APIs (`allow_private_destinations`) keep the hardened default: a publisher- or brand-supplied URL has no legitimate special-use destination, so no escape hatch is threaded to them. The `JwksFetcher` and `AsyncJwksFetcher` Protocols are unchanged. Widening them breaks every custom fetcher an adopter has written, and the built-in fetchers reaching the hardened default is the behavior they should have. An adopter needing the relaxation on its own transport calls `resolve_and_validate_host` or `build_ip_pinned_transport` directly, which is what the custom-fetcher section of the package docstring already points at. The package docstring's custom-fetcher section names both gates and their defaults, next to the primitives it lists. ## Why API DESIGN and not the ranges Nothing is mis-classified. `_EXTRA_BLOCKED_NETWORKS` is complete, the six `ipaddress` flags are the right predicate, and the empirical verification in that table's comment holds. The defect is the scope of one parameter, so the change is to the parameter. A narrower fix — making `_EXTRA_BLOCKED_NETWORKS` unconditional and leaving the flags behind `allow_private` — closes six of the 23 named ranges and leaves 17 open, because `ipaddress.is_private` is itself far wider than RFC 1918: it reports True for documentation, benchmarking, protocol-assignment, 6to4, Teredo and `240.0.0.0/4` space. Any gate keyed on that flag admits all of it. Naming the private ranges positively is what makes the relaxation as narrow as the flag's name. ## Why breaking, and the migration `allow_private=True` refuses addresses it previously accepted, so this carries `!` and a `BREAKING CHANGE:` footer. A caller that relied on it to reach link-local, multicast, carrier-grade NAT, documentation or benchmarking space passes `allow_special_use=True` alongside it and keeps the previous behavior exactly. The alternative — adding `allow_special_use` while leaving `allow_private` relaxing everything — is additive and fixes nothing: the hole stays open for every existing caller, and the new parameter has no effect unless `allow_private` is already off. The behavior change is the fix. Callers using the flag for what it is documented for are unaffected. `DockerLocalhostRewrite`, whose `allow_private_destinations=True` requirement is documented in `docs/webhooks/migration-from-fragmented-senders.md`, rewrites to `host.docker.internal` or `172.17.0.1`; both resolve into RFC 1918 space and keep working. ## Tests `tests/conformance/signing/test_jwks.py`, table-driven over the 23 named ranges so a future relaxation fails per range rather than in aggregate: * `test_ssrf_special_use_ranges_stay_blocked_under_allow_private` — each range refused with `allow_private=True`. * `test_ssrf_special_use_ranges_admitted_only_by_their_own_flag` — each range admitted with `allow_special_use=True`, which is the migration path. * `test_ssrf_private_destinations_honour_allow_private` — RFC 1918, RFC 4193 and loopback still admitted by `allow_private`, still refused without it. * `test_private_destination_networks_are_a_subset_of_the_base_policy` — walks `_PRIVATE_DESTINATION_NETWORKS` and asserts the base policy blocks every bound. This is what keeps the default posture identical to before the split: a range added to the private set that the union does not cover turns a refusal into an acceptance for `allow_private=False`. * `test_private_and_special_use_sets_are_disjoint` — one address, one gate. * `test_ssrf_metadata_blocked_under_both_relaxations` — every entry in `BLOCKED_METADATA_IPS` refused under `allow_private`, under `allow_special_use`, and under both. Each one sits inside a range one of the gates admits (`169.254.169.254` in link-local, `100.100.100.200` in RFC 6598, `192.0.0.192` in RFC 6890, `fd00:ec2::254` in RFC 4193), so the unconditional check has to run ahead of both. Moving it behind them fails all four cases. The two tests that asserted the old behavior, `test_ssrf_6to4_relay_honours_allow_private_override` and `test_ssrf_cgnat_honours_allow_private_override`, are replaced by the rows above; both ranges now appear in the stay-blocked table. Verified against the pinned base: 50 of the new parametrized cases fail without the change and pass with it. `make lint typecheck validate-generated test-type-checks check-type-ignore-contract` pass. `tests/conformance/signing/`, `tests/test_adagents.py`, `tests/test_canonical_reference_resolver.py`, `tests/test_a2a_push_security.py`, the webhook-delivery and webhook-destination-policy suites, and every test naming `allow_private` or `allow_private_destinations` pass. ## What an adopter can now do Ask for private destinations without asking for anything else: ```python from adcp.signing import resolve_and_validate_host # A test dialling its own loopback origin. resolve_and_validate_host("https://127.0.0.1:8443/webhook", allow_private=True) # The same posture refuses a buyer-supplied URL that resolves into shared # carrier space, which allow_private previously accepted. resolve_and_validate_host("https://buyer.example/webhook", allow_private=True) # SSRFValidationError: resolved IP 100.64.0.1 is in a reserved range # (IANA special-use; allow_private does not admit it) ``` Before this change the only way to express that was to re-enumerate the ranges outside the SDK: `_EXTRA_BLOCKED_NETWORKS` and `BLOCKED_METADATA_IPS` are module-private and absent from `adcp.signing.__all__`, so an adopter could not even reuse the tables it was duplicating. The refusal message names the gate that admits the address, so an operator hitting it in a dev deployment reads which knob to turn. BREAKING CHANGE: `allow_private=True` now admits only RFC 1918, RFC 4193 unique-local and loopback destinations. The IANA special-use ranges it used to admit — link-local, multicast, the unspecified address, RFC 6598 carrier-grade NAT, documentation and benchmarking space, RFC 6890 protocol assignments, 240.0.0.0/4, 6to4, Teredo, NAT64, the IANA anycast and identifier ranges and unallocated IPv6 — are refused unless the caller also passes the new `allow_special_use=True`; cloud metadata addresses stay refused under both. Direct callers of `validate_resolved_ip`, `resolve_and_validate_host`, `validate_jwks_uri`, `build_ip_pinned_transport`, `build_async_ip_pinned_transport` or the deprecated `abuild_ip_pinned_transport` that dial those ranges migrate by adding `allow_special_use=True` next to `allow_private=True`. The higher-level `allow_private_destinations=True` surfaces — webhook delivery and `WebhookSender`, the audit sink, agent and brand.json resolution, and the built-in JWKS and revocation fetchers — forward only `allow_private` and get no special-use relaxation: deployments that reached a special-use address through them must move the destination into RFC 1918, RFC 4193 or loopback space, or supply their own client or transport built with `allow_special_use=True` where that surface accepts one. Loopback, RFC 1918 and RFC 4193 destinations, including the `DockerLocalhostRewrite` targets, are unaffected.
1 parent a0fbee3 commit a287ee2

4 files changed

Lines changed: 289 additions & 30 deletions

File tree

‎src/adcp/signing/__init__.py‎

Lines changed: 12 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -90,6 +90,18 @@
9090
* :func:`validate_uri_static` / :func:`validate_resolved_ip` — the
9191
DNS-free registration check and resolved-address policy as separate
9292
primitives for applications that resolve only when they dial.
93+
94+
Each of those that inspects an address — everything above except
95+
:func:`validate_uri_static`, which is the DNS-free scheme, host and
96+
port half — takes two independent relaxations, both off by default.
97+
``allow_private`` admits RFC 1918, RFC 4193 unique-local and loopback
98+
destinations: a test's own origin, a container bridge, an on-prem
99+
service. ``allow_special_use`` admits the rest of the blocked set,
100+
among it link-local, multicast, the unspecified address, documentation
101+
and benchmarking space, RFC 6598 carrier-grade NAT, and the IANA
102+
anycast and identifier ranges. Enabling private destinations leaves the
103+
special-use ranges refused. Cloud metadata endpoints are refused under
104+
both.
93105
"""
94106

95107
from __future__ import annotations

‎src/adcp/signing/ip_pinned_transport.py‎

Lines changed: 6 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -293,6 +293,7 @@ def build_ip_pinned_transport(
293293
uri: str,
294294
*,
295295
allow_private: bool = False,
296+
allow_special_use: bool = False,
296297
allowed_ports: frozenset[int] | None = None,
297298
verify: bool = True,
298299
) -> IpPinnedTransport:
@@ -317,6 +318,7 @@ def build_ip_pinned_transport(
317318
hostname, resolved_ip, _port = resolve_and_validate_host(
318319
uri,
319320
allow_private=allow_private,
321+
allow_special_use=allow_special_use,
320322
allowed_ports=allowed_ports,
321323
)
322324
return IpPinnedTransport(hostname=hostname, resolved_ip=resolved_ip, verify=verify)
@@ -326,6 +328,7 @@ def build_async_ip_pinned_transport(
326328
uri: str,
327329
*,
328330
allow_private: bool = False,
331+
allow_special_use: bool = False,
329332
allowed_ports: frozenset[int] | None = None,
330333
verify: bool = True,
331334
) -> AsyncIpPinnedTransport:
@@ -342,6 +345,7 @@ def build_async_ip_pinned_transport(
342345
hostname, resolved_ip, _port = resolve_and_validate_host(
343346
uri,
344347
allow_private=allow_private,
348+
allow_special_use=allow_special_use,
345349
allowed_ports=allowed_ports,
346350
)
347351
return AsyncIpPinnedTransport(hostname=hostname, resolved_ip=resolved_ip, verify=verify)
@@ -351,6 +355,7 @@ def abuild_ip_pinned_transport(
351355
uri: str,
352356
*,
353357
allow_private: bool = False,
358+
allow_special_use: bool = False,
354359
allowed_ports: frozenset[int] | None = None,
355360
verify: bool = True,
356361
) -> AsyncIpPinnedTransport:
@@ -371,6 +376,7 @@ def abuild_ip_pinned_transport(
371376
return build_async_ip_pinned_transport(
372377
uri,
373378
allow_private=allow_private,
379+
allow_special_use=allow_special_use,
374380
allowed_ports=allowed_ports,
375381
verify=verify,
376382
)

‎src/adcp/signing/jwks.py‎

Lines changed: 118 additions & 18 deletions
Original file line numberDiff line numberDiff line change
@@ -106,6 +106,51 @@
106106
ipaddress.ip_network("2001:20::/28"),
107107
)
108108

109+
# The ranges ``allow_private`` admits, and the only ones it admits.
110+
#
111+
# ``allow_private`` exists so a deployment can dial a destination on a network
112+
# the operator controls: a test's own loopback origin, a container-bridge
113+
# address, an on-prem service on RFC 1918 or RFC 4193 space. Those are the
114+
# ranges below.
115+
#
116+
# Every other range the policy blocks stays blocked under ``allow_private``
117+
# and moves behind ``allow_special_use``. The distinction is reachability
118+
# intent, not classification accident:
119+
#
120+
# * ``ipaddress.is_private`` is a far wider predicate than RFC 1918. On every
121+
# supported interpreter it also reports True for, among others, RFC 5737
122+
# documentation space (``192.0.2.0/24``, ``198.51.100.0/24``,
123+
# ``203.0.113.0/24``), RFC 2544 benchmarking (``198.18.0.0/15``),
124+
# RFC 6890 IETF protocol assignments (``192.0.0.0/24``), ``240.0.0.0/4``,
125+
# RFC 3849 ``2001:db8::/32``, 6to4 ``2002::/16`` and Teredo ``2001::/32``.
126+
# None of those is a private destination an operator deploys a service to.
127+
# * ``is_multicast``, ``is_reserved``, ``is_unspecified`` and the
128+
# ``_EXTRA_BLOCKED_NETWORKS`` anycast and identifier ranges are never a
129+
# unicast destination at all. ``is_reserved`` alone covers unallocated IPv6
130+
# space (``400::/8`` and up), which is why the special-use half is DERIVED
131+
# as "blocked by the base policy, not a private destination" rather than
132+
# enumerated — an enumeration cannot be complete, and a range CPython
133+
# reclassifies later lands on the right side without an edit here.
134+
# ``100.64.0.0/10`` in particular is a range AdCP names in the deny list a
135+
# fetcher MUST apply (spec 3.1.1, "Webhook URL validation (SSRF)", step 2).
136+
# * Link-local (``169.254.0.0/16``, ``fe80::/10``) is autoconfiguration and
137+
# metadata space — three of the four entries in
138+
# :data:`BLOCKED_METADATA_IPS` live in or beside it — so it sits with the
139+
# special-use ranges rather than with the private ones.
140+
#
141+
# Membership is a subset of what the base policy below blocks, so
142+
# ``allow_private=False`` refuses exactly what it refused before this split.
143+
# ``test_private_destination_networks_are_a_subset_of_the_base_policy`` pins
144+
# that.
145+
_PRIVATE_DESTINATION_NETWORKS: tuple[ipaddress.IPv4Network | ipaddress.IPv6Network, ...] = (
146+
ipaddress.ip_network("10.0.0.0/8"), # RFC 1918
147+
ipaddress.ip_network("172.16.0.0/12"), # RFC 1918
148+
ipaddress.ip_network("192.168.0.0/16"), # RFC 1918
149+
ipaddress.ip_network("127.0.0.0/8"), # RFC 1122 loopback
150+
ipaddress.ip_network("fc00::/7"), # RFC 4193 unique local
151+
ipaddress.ip_network("::1/128"), # RFC 4291 loopback
152+
)
153+
109154
# Recommended destination ports for hardened SSRF-validated outbound HTTP
110155
# deployments. AdCP itself does not constrain ``pushNotificationConfig.url``
111156
# ports (see ``schemas/cache/core/push-notification-config.json``), so the
@@ -210,6 +255,7 @@ def validate_jwks_uri(
210255
uri: str,
211256
*,
212257
allow_private: bool = False,
258+
allow_special_use: bool = False,
213259
allowed_ports: frozenset[int] | None = None,
214260
) -> None:
215261
"""Raise SSRFValidationError on blocked IP, bad scheme, or disallowed port.
@@ -218,7 +264,12 @@ def validate_jwks_uri(
218264
:func:`resolve_and_validate_host` returns the accepted IP when the
219265
caller needs it for IP-pinned connects.
220266
"""
221-
resolve_and_validate_host(uri, allow_private=allow_private, allowed_ports=allowed_ports)
267+
resolve_and_validate_host(
268+
uri,
269+
allow_private=allow_private,
270+
allow_special_use=allow_special_use,
271+
allowed_ports=allowed_ports,
272+
)
222273

223274

224275
def validate_uri_static(
@@ -268,37 +319,78 @@ def _normalize_resolved_ip(
268319
return ip
269320

270321

322+
def _is_blocked_by_base_policy(ip: ipaddress.IPv4Address | ipaddress.IPv6Address) -> bool:
323+
"""True when the address is outside the set a hardened fetcher may dial.
324+
325+
The union the policy has always refused: the six ``ipaddress`` flags plus
326+
the ranges those flags miss. Spelled once so the two relaxation gates
327+
partition this set instead of each restating part of it.
328+
"""
329+
return (
330+
ip.is_private
331+
or ip.is_loopback
332+
or ip.is_link_local
333+
or ip.is_multicast
334+
or ip.is_reserved
335+
or ip.is_unspecified
336+
or any(ip in network for network in _EXTRA_BLOCKED_NETWORKS)
337+
)
338+
339+
340+
def _is_private_destination(ip: ipaddress.IPv4Address | ipaddress.IPv6Address) -> bool:
341+
"""True for a destination on a network the operator controls.
342+
343+
See :data:`_PRIVATE_DESTINATION_NETWORKS` for the ranges and why the
344+
rest of the blocked set is not among them.
345+
"""
346+
return any(ip in network for network in _PRIVATE_DESTINATION_NETWORKS)
347+
348+
271349
def validate_resolved_ip(
272350
ip: ipaddress.IPv4Address | ipaddress.IPv6Address,
273351
*,
274352
allow_private: bool = False,
353+
allow_special_use: bool = False,
275354
) -> None:
276355
"""Validate one already-resolved address against the shared SSRF policy.
277356
278-
Cloud metadata endpoints are always rejected. Private and special-use
279-
ranges are rejected unless ``allow_private`` is enabled. IPv4-mapped IPv6
280-
addresses are unwrapped internally so behavior is identical on Python
281-
3.10 and newer.
357+
Cloud metadata endpoints are always rejected. The rest of the blocked set
358+
splits in two, and each half has its own gate:
359+
360+
* **Private destinations** — RFC 1918, RFC 4193 unique-local, and
361+
loopback. ``allow_private`` admits these, which is what lets a test
362+
dial its own loopback origin or a container-bridge address.
363+
* **Special-use ranges** — everything else the policy refuses:
364+
link-local, multicast, unspecified, documentation and benchmarking
365+
space, RFC 6598 carrier-grade NAT, and the IANA anycast and identifier
366+
ranges in :data:`_EXTRA_BLOCKED_NETWORKS`. ``allow_special_use`` admits
367+
these. None is a destination an operator deploys a service to, so
368+
enabling private addresses leaves them refused.
369+
370+
IPv4-mapped IPv6 addresses are unwrapped internally so behavior is
371+
identical on Python 3.10 and newer.
282372
"""
283373
ip = _normalize_resolved_ip(ip)
284374
if str(ip) in BLOCKED_METADATA_IPS:
285375
raise SSRFValidationError(f"cloud metadata IP {ip} blocked")
286-
if not allow_private and (
287-
ip.is_private
288-
or ip.is_loopback
289-
or ip.is_link_local
290-
or ip.is_multicast
291-
or ip.is_reserved
292-
or ip.is_unspecified
293-
or any(ip in network for network in _EXTRA_BLOCKED_NETWORKS)
294-
):
295-
raise SSRFValidationError(f"resolved IP {ip} is in a reserved range")
376+
if not _is_blocked_by_base_policy(ip):
377+
return
378+
if _is_private_destination(ip):
379+
if not allow_private:
380+
raise SSRFValidationError(f"resolved IP {ip} is in a reserved range")
381+
return
382+
if not allow_special_use:
383+
raise SSRFValidationError(
384+
f"resolved IP {ip} is in a reserved range (IANA special-use; "
385+
f"allow_private does not admit it)"
386+
)
296387

297388

298389
def resolve_and_validate_host(
299390
uri: str,
300391
*,
301392
allow_private: bool = False,
393+
allow_special_use: bool = False,
302394
allowed_ports: frozenset[int] | None = None,
303395
) -> tuple[str, str, int]:
304396
"""Resolve the URI's hostname once and return ``(hostname, ip, port)``.
@@ -318,8 +410,14 @@ def resolve_and_validate_host(
318410
uri:
319411
A full URL. Only ``http`` and ``https`` schemes are accepted.
320412
allow_private:
321-
Skip the reserved-range check. For tests only; cloud-metadata
322-
IPs remain blocked unconditionally.
413+
Admit RFC 1918, RFC 4193 unique-local and loopback destinations.
414+
For tests and on-prem deployments; cloud-metadata IPs and the
415+
special-use ranges remain blocked.
416+
allow_special_use:
417+
Admit the rest of the blocked set — link-local, multicast,
418+
unspecified, documentation and benchmarking space, RFC 6598
419+
carrier-grade NAT, and the IANA anycast and identifier ranges.
420+
Cloud-metadata IPs remain blocked unconditionally.
323421
allowed_ports:
324422
Optional destination-port allowlist. ``None`` (default) imposes
325423
no port filter — the URL's port is unrestricted. Hardened
@@ -365,7 +463,9 @@ def resolve_and_validate_host(
365463
# per pyproject.toml so we unwrap explicitly.
366464
ip = _normalize_resolved_ip(ip)
367465
try:
368-
validate_resolved_ip(ip, allow_private=allow_private)
466+
validate_resolved_ip(
467+
ip, allow_private=allow_private, allow_special_use=allow_special_use
468+
)
369469
except SSRFValidationError as exc:
370470
last_rejection = str(exc)
371471
# Historical behavior of validate_jwks_uri was to raise on

0 commit comments

Comments
 (0)