Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
3 changes: 3 additions & 0 deletions conformance/CHECKSUMS
Original file line number Diff line number Diff line change
@@ -0,0 +1,3 @@
6e71737bd26abcd7106758de38316f477cfc69e792c153ac4d4ceca547841aba discover-resilience.json
5365bc6f7e61384ad71fd25543c0f70cfe12fb1fe156a5fc0aeb91768871e6e6 negotiable-floor.json
749b2ca0f17f52e6b0fb81ce0c2a6b0a0f1bc45f20c2848ad8cf582893fa0513 price-tag.json
133 changes: 133 additions & 0 deletions conformance/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,133 @@
# Port-drift conformance check

This directory is the **cross-port conformance suite** for the three Lightning
Enable Agent SDK ports:

- Python — `le-agent-sdk` (`F:\le-agent-sdk-python`, default branch `master`)
- .NET — `LightningEnable.AgentSdk` (`F:\le-agent-sdk-dotnet`, default branch `main`)
- TypeScript — `le-agent-sdk` (`F:\le-agent-sdk-ts`, default branch `main`)

## Why this exists (the oracle)

An audit of the three ports found that **wherever the ports of a shared behavior
disagreed, at least two of the three were wrong.** This held on every divergence
found (auth bypass, preimage fabrication, `max_amount` gap, ratings filter, the
`#41` price-tag divergence, the `#61` negotiable-floor divergence).

That observation is turned into a CI-enforced check here: for each
security-critical shared behavior we define **golden vectors** (input -> expected
output) in language-neutral JSON, and each port ships a **conformance test** that
runs those same vectors through *its own* implementation. If any port diverges,
that port's own CI goes red. Drift is caught automatically instead of by manual
cross-reading.

## What is covered (first version)

| Vectors file | Behavior | Entry point |
|---|---|---|
| `vectors/price-tag.json` | Parsing a capability `price` tag amount | `AgentCapability.fromNostrEvent` / `from_nostr_event` / `FromNostrEvent` |
| `vectors/negotiable-floor.json` | Parsing the `["negotiable","floor","<amount>"]` branch | same |
| `vectors/discover-resilience.json` | `discover()` batch resilience to one malformed payload (ledger #41) | `AgentManager.discover` + each port's relay-ingest layer |

Each vectors file is self-describing: it names the behavior, the entry point, the
outcome vocabulary, and every vector's expected outcome.

### Findings this suite encodes

- **Price parsing agrees across all three ports.** Valid amounts parse; `abc`,
`10.5`, `100abc` are rejected (throw); a bare `["price"]` records no price; and
`0` is valid (a free service). A **negative** amount (`-5`) was originally
**accepted by all three** and flagged in `price-tag.json` as an open
`designQuestion`. That question is now **decided (ledger #69, 2026-07-22):
a negative price/floor is rejected** — it is never meaningful and accepting it
is a fail-open smell, so it is treated like any other malformed amount (throw ->
the event is skipped). The golden now REJECTS `-5` (`negative-amount-rejected`)
and pins `0` as valid (`zero-accepted`); all three ports were tightened to
conform.
- **Negotiable-floor did NOT agree.** Python and .NET reject a malformed floor
amount (throw -> the event is skipped). TypeScript used `parseInt()`, which
returns `NaN` for `"abc"` and silently truncates `"10.5"`->`10` /
`"100abc"`->`100`, and never throws. Two ports reject, one keeps a bogus value:
by the oracle, the one that keeps it is the bug (ledger #61). The golden REJECTS
a malformed floor and the TypeScript port was fixed to conform (a `NaN` floor is
worse than useless: every price-floor comparison against `NaN` is false, so a
malformed floor silently passes downstream instead of being rejected). The floor
amount is parsed with the **same** non-negative-integer rules as the price
amount, so a **negative** floor is rejected too (ledger #69; golden
`negative-floor-rejected`, with `0` pinned valid by `zero-floor-accepted`).

## How each port wires it into CI

The conformance test is an ordinary test file, so each repo's existing test job
picks it up with no workflow change:

- **Python** — `tests/test_conformance.py`, run by `pytest tests/` (`.github/workflows/test.yml`).
- **.NET** — `tests/LightningEnable.AgentSdk.Tests/ConformanceTests.cs`, run by
`dotnet test`. The vectors are linked into the test project and copied next to
the test DLL (`<None Include="..\..\conformance\vectors\*.json" CopyToOutputDirectory="PreserveNewest" />`).
- **TypeScript** — `tests/conformance.test.ts`, run by `vitest run` (`npm test`).

Each port reads the JSON from this canonical directory (`conformance/vectors/`) at
the repo root, so the test and the vectors can never point at stale copies.

## How the vectors stay in sync across the three repos

These are three separate repositories (no monorepo), so the vectors are
**physically copied** into each one at the same path. Two things keep the copies
honest:

1. **A single source of truth.** The canonical copy lives in the **Python** repo
(`le-agent-sdk-python/conformance/`). To change a vector, edit it there, then
copy `conformance/vectors/*.json` verbatim into the other two repos in the same
change set.
2. **A shared checksum guard.** `conformance/CHECKSUMS` lists the SHA-256 of each
vectors file and is **byte-identical in all three repos**. Every port's
conformance test recomputes the checksums of its local vectors (over
**LF-normalized** bytes, so CRLF checkouts on Windows CI don't matter) and
asserts they match `CHECKSUMS`. Because the same `CHECKSUMS` constant is present
in every repo and each repo's JSON must match it, the JSON is transitively
identical across all three. If someone edits a vector in one repo only, that
repo's checksum test fails; if they also update `CHECKSUMS` but forget a repo,
the forgotten repo fails. Either way CI catches the drift.

Regenerate `CHECKSUMS` after changing any vector:

```sh
# from the repo root, on any port
cd conformance
python - <<'PY'
import hashlib, pathlib
lines = []
for p in sorted(pathlib.Path("vectors").glob("*.json")):
data = p.read_bytes().replace(b"\r\n", b"\n").replace(b"\r", b"\n")
lines.append(f"{hashlib.sha256(data).hexdigest()} {p.name}")
pathlib.Path("CHECKSUMS").write_text("\n".join(lines) + "\n")
print("\n".join(lines))
PY
```

## How to extend

**Add a behavior to this suite:**

1. Add a `vectors/<behavior>.json` file (same self-describing shape: `behavior`,
`entrypoint`, an outcome vocabulary, and a `vectors`/`scenarios` array).
2. First DISCOVER what each port actually does for every vector (read the code /
run it). Where all three agree, that agreed output is the golden. Where they
diverge: if the correct behavior is unambiguous, set the golden to it and FIX
the outlier port (failing-vector-first: make it red, fix, green); if it is a
real design question, set the golden to the current agreement and surface the
question rather than picking a side.
3. Add an assertion block for the new file to each port's conformance test.
4. Regenerate `CHECKSUMS` and copy the new vectors + `CHECKSUMS` into all three
repos in one change set.

**Extend to the `l402-*` three-port trio:** the same pattern transplants directly.
The `l402-client-*` libraries share security-critical parsing too — bolt11 invoice
amount decoding, `max_amount` budget enforcement, the L402 `WWW-Authenticate`
challenge parse. Stand up a parallel `conformance/` in the `l402-*` repos with
`vectors/invoice-amount.json`, `vectors/max-amount-budget.json`,
`vectors/l402-challenge.json`, and the identical CHECKSUMS-guard + single-source
copy mechanism. Candidate next behaviors already visible in these SDK ports (all
still using bare `parseInt`/`int()` and therefore worth pinning): budget-tag,
attestation-rating, and request-expiration parsing.
25 changes: 25 additions & 0 deletions conformance/vectors/discover-resilience.json
Original file line number Diff line number Diff line change
@@ -0,0 +1,25 @@
{
"behavior": "discover-batch-resilience",
"reference": "le-agent-sdk ledger #41",
"entrypoint": "AgentManager.discover (per-event), plus each port's relay-ingest layer",
"description": "A discovery batch of [valid, malformed, valid] relay payloads MUST return the valid capabilities. One malformed or hostile payload must never abort the batch and take every valid capability down with it (a single-hostile-relay DoS). The skip must fail closed and LOUDLY (a warning naming the offending payload), never silently. Each malformed kind is dropped at the layer where a given port first meets it (parse, authenticity, or relay-frame merge); the shared contract is the scenario set + expectedSurvivors, and each port realizes it at its own layer (see conformance/README.md).",
"batchLayout": ["valid", "malformed", "valid"],
"expectedSurvivors": 2,
"scenarios": [
{
"name": "bad-price",
"malformed": "a capability event whose price tag amount is unparseable ([\"price\",\"abc\"])",
"droppedAt": "per-event capability parse (fromNostrEvent throws, discover skips just that event)"
},
{
"name": "missing-committed-field",
"malformed": "a relay payload missing / mistyping a field committed by the Nostr event id, so it cannot be authenticated or frame-parsed",
"droppedAt": "authenticity check (py/ts) or wire-frame parse (.NET TryParseEventMessage) — before the payload is trusted"
},
{
"name": "non-dict-payload",
"malformed": "a relay payload that is not an event object at all (e.g. a bare string)",
"droppedAt": "relay merge (py/ts queryRelays) or wire-frame parse (.NET TryParseEventMessage)"
}
]
}
49 changes: 49 additions & 0 deletions conformance/vectors/negotiable-floor.json
Original file line number Diff line number Diff line change
@@ -0,0 +1,49 @@
{
"behavior": "negotiable-floor-parsing",
"reference": "le-agent-sdk ledger #61 (typescript floor uses parseInt -> keeps NaN/truncated), #69 (reject negative amount)",
"entrypoint": "AgentCapability.fromNostrEvent (ts) / from_nostr_event (py) / FromNostrEvent (.NET)",
"description": "How each port parses the ['negotiable','floor','<amount>'] branch of a capability. The floor amount MUST be a non-negative integer, parsed with the SAME rules as the price-tag amount. python and .NET reject a malformed floor amount (throw -> the whole event is skipped by discover). typescript used parseInt(), which returns NaN for 'abc' and silently truncates '10.5'->10 / '100abc'->100 and NEVER throws, so it kept a bogus minPriceSats. The golden REJECTS a malformed floor AND a negative floor: minPriceSats must never be NaN/truncated/negative, because a NaN floor passes every price-floor comparison downstream (every comparison against NaN is false) and a negative floor is meaningless. Zero is valid (negotiable down to free).",
"outcomes": {
"ok": "Parse succeeds; 'negotiable' and 'minPriceSats' (nullable) equal the expected values.",
"reject": "The capability-parse entrypoint throws (ValueError / Error / FormatException)."
},
"vectors": [
{
"name": "valid-floor",
"tags": [["d", "conformance"], ["negotiable", "floor", "5000"]],
"expect": { "outcome": "ok", "negotiable": true, "minPriceSats": 5000 }
},
{
"name": "zero-floor-accepted",
"note": "A floor of zero is valid (negotiable down to free). Pins that rejecting negatives (ledger #69) did not over-reject zero.",
"tags": [["d", "conformance"], ["negotiable", "floor", "0"]],
"expect": { "outcome": "ok", "negotiable": true, "minPriceSats": 0 }
},
{
"name": "negotiable-false",
"tags": [["d", "conformance"], ["negotiable", "false"]],
"expect": { "outcome": "ok", "negotiable": false, "minPriceSats": null }
},
{
"name": "reject-floor-non-numeric",
"tags": [["d", "conformance"], ["negotiable", "floor", "abc"]],
"expect": { "outcome": "reject" }
},
{
"name": "reject-floor-decimal",
"tags": [["d", "conformance"], ["negotiable", "floor", "10.5"]],
"expect": { "outcome": "reject" }
},
{
"name": "reject-floor-trailing-suffix",
"tags": [["d", "conformance"], ["negotiable", "floor", "100abc"]],
"expect": { "outcome": "reject" }
},
{
"name": "negative-floor-rejected",
"note": "Negatives are rejected by decision (ledger #69, 2026-07-22): a negative floor is never meaningful and is a fail-open smell, so it is treated like any other malformed amount (throw -> the event is skipped). Zero stays valid (see zero-floor-accepted).",
"tags": [["d", "conformance"], ["negotiable", "floor", "-5"]],
"expect": { "outcome": "reject" }
}
]
}
55 changes: 55 additions & 0 deletions conformance/vectors/price-tag.json
Original file line number Diff line number Diff line change
@@ -0,0 +1,55 @@
{
"behavior": "price-tag-parsing",
"reference": "le-agent-sdk ledger #41 (price-tag divergence), #69 (reject negative amount)",
"entrypoint": "AgentCapability.fromNostrEvent (ts) / from_nostr_event (py) / FromNostrEvent (.NET)",
"description": "How each port parses a capability 'price' tag amount. Driven through the public capability-parse entrypoint (not the lower-level fromTag/ParseSats helper) because that is the one uniform public surface across all three ports and the real production code path. A price amount MUST be a non-negative integer: a malformed amount (non-numeric / decimal / trailing-suffix) AND a negative amount are both rejected (throw), never silently coerced to NaN/0/truncated or accepted as-is, because a bogus or negative price sails through every downstream budget/affordability check. Zero is valid (a free service).",
"outcomes": {
"ok": "Parse succeeds; the first parsed price amount equals priceSats. 'unit'/'model' are asserted only by ports that model them (python, typescript); the .NET port stores a single integer PriceSats and ignores unit/model.",
"reject": "The capability-parse entrypoint throws (ValueError / Error / FormatException).",
"no-price": "Parse succeeds but records no price (python/typescript: empty pricing list; .NET: PriceSats == 0, its default)."
},
"vectors": [
{
"name": "valid-minimal",
"tags": [["d", "conformance"], ["price", "100"]],
"expect": { "outcome": "ok", "priceSats": 100, "unit": "sats", "model": "per-request" }
},
{
"name": "valid-unit-and-model",
"tags": [["d", "conformance"], ["price", "100", "msat", "per-call"]],
"expect": { "outcome": "ok", "priceSats": 100, "unit": "msat", "model": "per-call" }
},
{
"name": "zero-accepted",
"note": "Zero is a valid advertised price (a free service). Pins that rejecting negatives (ledger #69) did not over-reject zero.",
"tags": [["d", "conformance"], ["price", "0"]],
"expect": { "outcome": "ok", "priceSats": 0, "unit": "sats", "model": "per-request" }
},
{
"name": "reject-non-numeric",
"tags": [["d", "conformance"], ["price", "abc"]],
"expect": { "outcome": "reject" }
},
{
"name": "reject-decimal",
"tags": [["d", "conformance"], ["price", "10.5"]],
"expect": { "outcome": "reject" }
},
{
"name": "reject-trailing-suffix",
"tags": [["d", "conformance"], ["price", "100abc"]],
"expect": { "outcome": "reject" }
},
{
"name": "missing-amount",
"tags": [["d", "conformance"], ["price"]],
"expect": { "outcome": "no-price" }
},
{
"name": "negative-amount-rejected",
"note": "Negatives are rejected by decision (ledger #69, 2026-07-22): a negative advertised price is never meaningful and is a fail-open smell, so it is treated like any other malformed amount (throw -> the event is skipped). Was previously accepted by all three ports and flagged as an open designQuestion; the question is now decided. Zero stays valid (see zero-accepted).",
"tags": [["d", "conformance"], ["price", "-5"]],
"expect": { "outcome": "reject" }
}
]
}
22 changes: 20 additions & 2 deletions src/le_agent_sdk/models/capability.py
Original file line number Diff line number Diff line change
Expand Up @@ -6,6 +6,24 @@
from typing import Any, Optional


def _parse_sats_amount(value: str) -> int:
"""Parse a non-negative integer sats amount, or raise ``ValueError``.

A price/floor amount MUST be a non-negative integer. ``int()`` already rejects
non-numeric / decimal / trailing-suffix values (``ValueError``); this
additionally rejects a **negative** amount (ledger #69): a negative advertised
price/floor is never meaningful and accepting it is a fail-open smell. The
rejection raises the *same* ``ValueError`` as any other malformed amount so the
parse-error path (``discover()``'s per-event skip) treats it identically. Zero
is valid (a free service). Shared by BOTH the price-tag and negotiable-floor
parse so the two can never drift apart.
"""
amount = int(value)
if amount < 0:
raise ValueError(f"Sats amount must be non-negative, got: {value!r}")
return amount


@dataclass
class AgentPricing:
"""Pricing information for an agent capability."""
Expand All @@ -23,7 +41,7 @@ def from_tag(cls, tag: list[str]) -> AgentPricing:
"""Parse from a Nostr 'price' tag: ['price', amount, unit, model]."""
if len(tag) < 2:
raise ValueError(f"Invalid price tag: {tag}")
amount = int(tag[1])
amount = _parse_sats_amount(tag[1])
unit = tag[2] if len(tag) > 2 else "sats"
model = tag[3] if len(tag) > 3 else "per-request"
return cls(amount=amount, unit=unit, model=model)
Expand Down Expand Up @@ -92,7 +110,7 @@ def from_nostr_event(cls, event: dict[str, Any]) -> AgentCapability:
cap.negotiable = True
elif tag[1] == "floor" and len(tag) > 2:
cap.negotiable = True
cap.min_price_sats = int(tag[2])
cap.min_price_sats = _parse_sats_amount(tag[2])

return cap

Expand Down
Loading
Loading