diff --git a/README.md b/README.md index 459a5b8d..42a3314e 100644 --- a/README.md +++ b/README.md @@ -102,7 +102,7 @@ The SDK is **layered the same way in every language** — full detail in | **L2 · I/O** | the only tier that dials the network — key/endpoint/offer-key resolvers behind one SSRF-guarded client | `sdk/go/resolvers` | `ramp_sdk.resolvers` | `sdk/ts/resolvers` | | **L2 · transport** | transport-neutral composition + Connect bindings; the agent, broker and catalog clients | `sdk/go/core` · `connect` · `connectserver` | `ramp_sdk.core` · `server_verify` · `ramp_sdk.client` (+ `sync`) | `sdk/ts/core` · `hono` · `sdk/ts/client` | -The SDK is scored by integration role — edge, agent, account setup, publisher, exchange operator — and the agent and publisher roles ship in all three languages: the agent verbs (`discover` / `resolve` / `execute` / `reportUsage` / `dispute` / `fetch`) and the publisher verbs (`pushResources` / `removeResources` / `refreshCatalog`) plus the two-tier entry pre-check. Go is the reference/oracle; Python and TypeScript mirror it face-for-face. The exact +The SDK is scored by integration role — edge, agent, account setup, publisher, exchange operator — and the agent, account-setup and publisher roles ship in all three languages: the agent verbs (`discover` / `resolve` / `execute` / `reportUsage` / `dispute` / `fetch`), the account-setup verbs (`register` / `getAccountStatus`) with the fresh-manifest read of an Exchange's registration requirements, and the publisher verbs (`pushResources` / `removeResources` / `refreshCatalog`) plus the two-tier entry pre-check. Go is the reference/oracle; Python and TypeScript mirror it face-for-face. The exact public surface per language — every symbol and its cross-language counterpart, the documented divergences, and the conformance-vector replay coverage — is tracked in the **generated, CI-drift-gated** [SDK parity matrix](docs/sdk-parity-matrix.md). The diff --git a/conformance/errordetail_domain_test.go b/conformance/errordetail_domain_test.go new file mode 100644 index 00000000..79400524 --- /dev/null +++ b/conformance/errordetail_domain_test.go @@ -0,0 +1,216 @@ +package conformance + +// Drift guard: what an ErrorDetail's domain may name. +// +// `domain` is the one field of ErrorDetail the contract leaves open. It carries no +// protovalidate rule, the proto describes it only as "Stable grouping for the failing +// surface, e.g. ramp.v1.ExchangeService", and ADR-019 hedges the same way — an example, +// never a closed set. That is the right call for a field whose whole job is grouping for +// generic tooling: a rule enumerating legal values would have to move every time a tier +// does. +// +// It is not, however, a licence for anything. A grouping key groups nothing if two writers +// spell the same surface differently, or if one names a surface that does not exist, and +// both have happened. The values in the committed corpora fall into exactly two shapes, +// and the SUFFIX is what separates them: +// +// - `.Service` names an RPC service the contract DEFINES. An answer +// carrying ramp.v1.ExchangeService came from that service. +// - `.` names a TIER that is not a service and appears in no +// descriptor — ramp.v1.Edge for a delivery edge, ramp.v1.Client for a refusal the +// client computed before sending. Neither is a proto symbol, and neither should be: +// the surface that refused is a real thing whether or not the contract has a message +// for it. +// +// So the rule this holds is narrow and checkable: a Service-suffixed domain must name a +// service that exists, and a bare-noun domain must not name one. That catches the failure +// this guard was written for — a corpus fixture naming ramp.v1.RegistrationService, a +// service the contract has never defined, sitting beside two sibling vectors that use +// ramp.v1.ExchangeService for the same reason block. +// +// The values are READ from the SDK's committed vectors rather than restated here, for the +// reason ver_field_contract_test.go gives at length: nothing in conformance depends on +// sdk/, by design, since it is the guard tier BELOW the SDKs. A committed file generated +// from the real constructors, and already replayed by the Python and TypeScript parity +// suites, is a data read exactly like the corpus and the doc scans. + +import ( + "encoding/json" + "os" + "strings" + "testing" +) + +// The committed corpora that carry an ErrorDetail domain. Every one of them is emitted by +// a Go generator from the real builders and replayed by both ports. +const ( + errorDetailVectors = "../sdk/go/helpers/testdata/error-detail-vectors.json" + connectErrorVectors = "../sdk/go/connect/testdata/connect-error-vectors.json" + synthesizedDetailVectors = "../sdk/go/connect/testdata/synthesized-detail-vectors.json" +) + +// domainUse is one recorded domain and enough context to name it in a failure. +type domainUse struct { + Corpus string + Vector string + Domain string +} + +// recordedDomains reads every domain the corpora carry. A read or parse error is fatal +// rather than skipped: a missing file means this guard has lost its anchor, and passing +// silently would be worse than failing. +func recordedDomains(t *testing.T) []domainUse { + t.Helper() + var out []domainUse + + read := func(path string, into any) { + b, err := os.ReadFile(path) + if err != nil { + t.Fatalf("read %s: %v", path, err) + } + if err := json.Unmarshal(b, into); err != nil { + t.Fatalf("parse %s: %v", path, err) + } + } + + var details struct { + Vectors []struct{ Name, Domain string } `json:"vectors"` + } + read(errorDetailVectors, &details) + for _, v := range details.Vectors { + out = append(out, domainUse{errorDetailVectors, v.Name, v.Domain}) + } + + var envelopes struct { + Vectors []struct { + Name string `json:"name"` + Expect struct{ Domain string } `json:"expect"` + } `json:"vectors"` + } + read(connectErrorVectors, &envelopes) + for _, v := range envelopes.Vectors { + out = append(out, domainUse{connectErrorVectors, v.Name, v.Expect.Domain}) + } + + var synthesized struct { + Precheck []struct{ Name, Domain string } `json:"registration_precheck"` + Edge []struct{ Name, Domain string } `json:"edge_refusal"` + } + read(synthesizedDetailVectors, &synthesized) + for _, v := range synthesized.Precheck { + out = append(out, domainUse{synthesizedDetailVectors, v.Name, v.Domain}) + } + for _, v := range synthesized.Edge { + out = append(out, domainUse{synthesizedDetailVectors, v.Name, v.Domain}) + } + return out +} + +// contractServiceNames returns every service the contract defines, fully qualified. +// Walked from the descriptors through the one package list, so a contract package added +// there is covered without an edit here. +func contractServiceNames() map[string]bool { + out := map[string]bool{} + for _, fd := range ContractFiles() { + services := fd.Services() + for i := 0; i < services.Len(); i++ { + out[string(services.Get(i).FullName())] = true + } + } + return out +} + +// A domain either names a service the contract defines, or names a tier that is not one — +// and the Service suffix says which, so a reader can tell them apart without a lookup. +func TestErrorDetailDomain_SuffixSaysWhetherItNamesAService(t *testing.T) { + uses := recordedDomains(t) + if len(uses) == 0 { + t.Fatal("no domains were read — this guard would vacuously pass") + } + services := contractServiceNames() + if len(services) == 0 { + t.Fatal("the contract defines no services — this guard has lost its anchor") + } + + var checked int + for _, u := range uses { + if u.Domain == "" { + // Legal: a detail may carry no domain, and one vector records exactly that. + continue + } + checked++ + if !inAContractPackage(u.Domain) { + t.Errorf("%s/%s: domain %q is outside every contract package, so nothing "+ + "groups by it", u.Corpus, u.Vector, u.Domain) + continue + } + named := services[u.Domain] + switch { + case strings.HasSuffix(u.Domain, "Service") && !named: + t.Errorf("%s/%s: domain %q is spelled as a service and the contract defines "+ + "none by that name; either it names a real service or it drops the "+ + "suffix and names the tier that refused", + u.Corpus, u.Vector, u.Domain) + case !strings.HasSuffix(u.Domain, "Service") && named: + t.Errorf("%s/%s: domain %q names a service without saying so; the suffix is "+ + "what tells a reader an RPC service refused rather than a tier", + u.Corpus, u.Vector, u.Domain) + } + } + if checked == 0 { + t.Fatal("every recorded domain was empty — this guard would vacuously pass") + } +} + +// inAContractPackage reports whether a domain sits under one of the contract's packages. +// The tiers that are not services live there too: ramp.v1.Edge is not a proto symbol, but +// putting it anywhere else would make the namespace mean nothing. +func inAContractPackage(domain string) bool { + for _, pkg := range ContractPackages() { + if strings.HasPrefix(domain, pkg+".") && + !strings.Contains(strings.TrimPrefix(domain, pkg+"."), ".") { + return true + } + } + return false +} + +// The meta-tests: the detector answers for the shapes it claims to. Without these a typo +// in the suffix check reads as a pass. +func TestErrorDetailDomain_DetectorAnswersForBothShapes(t *testing.T) { + services := contractServiceNames() + + for _, name := range []string{"ramp.v1.ExchangeService", "ramp.v1.CatalogService"} { + if !services[name] { + t.Errorf("%s is not among the contract's services, so the guard's own "+ + "positive case is wrong", name) + } + } + for _, name := range []string{"ramp.v1.Edge", "ramp.v1.Client", "ramp.v1.RegistrationService"} { + if services[name] { + t.Errorf("%s is a contract service, which this guard assumes it is not", name) + } + } + for _, in := range []string{"ramp.v1.Edge", "ramp.admin.v1.AdminService"} { + if !inAContractPackage(in) { + t.Errorf("%q was read as outside every contract package", in) + } + } + for _, out := range []string{"", "Edge", "google.rpc.ErrorInfo", "ramp.v1.a.b"} { + if inAContractPackage(out) { + t.Errorf("%q was read as inside a contract package", out) + } + } +} + +// Guard against the descriptor walk quietly finding nothing, which would make every +// membership answer above vacuously false. +func TestErrorDetailDomain_ContractServicesAreReachable(t *testing.T) { + var walked int + for _, fd := range ContractFiles() { + walked += fd.Services().Len() + } + if walked == 0 { + t.Fatal("no services reached through ContractFiles()") + } +} diff --git a/conformance/retrieval_token_record_test.go b/conformance/retrieval_token_record_test.go new file mode 100644 index 00000000..25efa43c --- /dev/null +++ b/conformance/retrieval_token_record_test.go @@ -0,0 +1,204 @@ +package conformance + +// Drift guard: the delivery edge's refusal tokens, as the proto records them. +// +// A delivery edge is a code-capable worker with no protobuf runtime, so it answers a +// refused fetch with a string from its own vocabulary. RetrievalAuthFailureReason is the +// typed counterpart, and the mapping between them is recorded ONLY as a trailing comment +// beside each enum value: +// +// RETRIEVAL_AUTH_FAILURE_REASON_URL_EXPIRED = 1; // 'expired' +// +// That is a real contract — every SDK has to promote the same token to the same reason, or +// one language's caller branches on a typed reason another language's caller never gets — +// and a comment is a contract nothing enforces. So this reads the annotations out of the +// .proto SOURCE, the way ver_field_contract_test.go reads the rule that lives in a comment +// there, and holds the SDK's committed vectors to them. +// +// It is written because the record and the code came apart, silently and in one direction. +// Two of the three SDKs never read this table at all: they DERIVED each reason's name by +// uppercasing the token and prefixing it, which lands on the recorded value for two of the +// eleven tokens and on nothing for the rest. Nine refusals a real edge emits carried a +// typed reason in one language and none in the other two, and every suite was green, +// because no case compared a language's answer to the proto's record. +// +// The mapping is not derivable, which is the whole reason it is written down: 'expired' is +// URL_EXPIRED and 'pop_expired' is PROOF_EXPIRED, and neither is the enum suffix +// lowercased. A guard that recomputed the token from the value would reproduce the bug it +// exists to catch, so this compares against the source text and nothing else. +// +// Vectors are READ as data. Nothing in conformance depends on sdk/, by design — it is the +// guard tier BELOW the SDKs — and the corpus is a committed file generated from the real +// client and replayed by all three languages, exactly like the corpora the neighbouring +// guards read. + +import ( + "encoding/json" + "os" + "path/filepath" + "regexp" + "strings" + "testing" +) + +// retrievalReasonEnum is the enum whose values carry the token record. +const retrievalReasonEnum = "RetrievalAuthFailureReason" + +// tokenAnnotation matches one recorded value: the enum value, then the edge token in +// single quotes. A value with no quoted token — the UNSPECIFIED zero — does not match, and +// the completeness check below is what notices if a real value stops matching. +var tokenAnnotation = regexp.MustCompile( + `^\s*(RETRIEVAL_AUTH_FAILURE_REASON_[A-Z0-9_]+)\s*=\s*\d+;\s*//\s*'([a-z0-9_]+)'`) + +// recordedTokens reads the token→value record out of the .proto source, as +// value→token and token→values. +func recordedTokens(t *testing.T) (byValue map[string]string, byToken map[string][]string) { + t.Helper() + byValue, byToken = map[string]string{}, map[string][]string{} + for _, cf := range Contract { + path := filepath.Join("..", "proto", cf.File.Path()) + b, err := os.ReadFile(path) + if err != nil { + t.Fatalf("read %s: %v", path, err) + } + for _, line := range strings.Split(string(b), "\n") { + m := tokenAnnotation.FindStringSubmatch(line) + if m == nil { + continue + } + byValue[m[1]] = m[2] + byToken[m[2]] = append(byToken[m[2]], m[1]) + } + } + return byValue, byToken +} + +// retrievalReasonValues returns every non-zero value of the enum, from the descriptor. +func retrievalReasonValues(t *testing.T) []string { + t.Helper() + var out []string + for _, fd := range ContractFiles() { + enums := fd.Enums() + for i := 0; i < enums.Len(); i++ { + ed := enums.Get(i) + if ed.Name() != retrievalReasonEnum { + continue + } + values := ed.Values() + for j := 0; j < values.Len(); j++ { + vd := values.Get(j) + if vd.Number() != 0 { + out = append(out, string(vd.Name())) + } + } + } + } + return out +} + +// Every reason the enum defines names the token it stands for. A value with no annotation +// is a token an SDK has no way to promote, which is the gap silently rather than a +// decision. +func TestRetrievalTokenRecord_EveryReasonNamesItsToken(t *testing.T) { + byValue, _ := recordedTokens(t) + values := retrievalReasonValues(t) + if len(values) == 0 { + t.Fatalf("%s defines no non-zero values — this guard has lost its anchor", + retrievalReasonEnum) + } + for _, v := range values { + if byValue[v] == "" { + t.Errorf("%s records no edge token; the mapping is not derivable from the "+ + "value's own spelling, so an unannotated value cannot be promoted at all", v) + } + } +} + +// The SDK's committed answers ARE the proto's record. Read from the corpus every language +// replays, so a port that computes the mapping instead of reading it fails here as well as +// in its own suite. +func TestRetrievalTokenRecord_TheCorpusMatchesTheProto(t *testing.T) { + _, byToken := recordedTokens(t) + if len(byToken) == 0 { + t.Fatal("no token annotations were read — this guard would vacuously pass") + } + + b, err := os.ReadFile(synthesizedDetailVectors) + if err != nil { + t.Fatalf("read %s: %v", synthesizedDetailVectors, err) + } + var doc struct { + Edge []struct { + Name string `json:"name"` + ReasonToken string `json:"reason_token"` + ReasonEnum string `json:"reason_enum"` + } `json:"edge_refusal"` + } + if err := json.Unmarshal(b, &doc); err != nil { + t.Fatalf("parse %s: %v", synthesizedDetailVectors, err) + } + if len(doc.Edge) == 0 { + t.Fatalf("%s records no edge refusals — this guard would vacuously pass", + synthesizedDetailVectors) + } + + recorded := map[string]bool{} + for _, row := range doc.Edge { + recorded[row.ReasonToken] = true + values := byToken[row.ReasonToken] + switch { + case len(values) == 0: + // A token the proto does not record. It must reach the caller untyped: + // promoting it would attribute a reason the contract never assigned. + if row.ReasonEnum != "" { + t.Errorf("%s: %q is not a token the proto records, but the SDK promoted "+ + "it to %s", row.Name, row.ReasonToken, row.ReasonEnum) + } + case len(values) > 1: + // Recorded against more than one value: the body does not say which check + // ran, so no language may choose. + if row.ReasonEnum != "" { + t.Errorf("%s: the proto records %q against %d values, so promoting it to "+ + "%s attributes a failure the wire never did", + row.Name, row.ReasonToken, len(values), row.ReasonEnum) + } + case row.ReasonEnum != values[0]: + t.Errorf("%s: the proto records %q as %s, the SDK answers %s", + row.Name, row.ReasonToken, values[0], row.ReasonEnum) + } + } + + // Completeness. A per-token table is only as good as its coverage, and a sample is + // what let two ports type two tokens and drop nine while looking correct. + for token := range byToken { + if !recorded[token] { + t.Errorf("the proto records the token %q and no vector replays it, so no "+ + "language is held to it", token) + } + } +} + +// The meta-tests: the annotation reader answers for the shapes it claims to. A regex that +// silently matches nothing is a guard that silently passes. +func TestRetrievalTokenRecord_ReaderAnswersForBothShapes(t *testing.T) { + for line, want := range map[string][2]string{ + " RETRIEVAL_AUTH_FAILURE_REASON_URL_EXPIRED = 1; // 'expired'": { + "RETRIEVAL_AUTH_FAILURE_REASON_URL_EXPIRED", "expired"}, + " RETRIEVAL_AUTH_FAILURE_REASON_PROOF_EXPIRED = 11; // 'pop_expired'": { + "RETRIEVAL_AUTH_FAILURE_REASON_PROOF_EXPIRED", "pop_expired"}, + } { + m := tokenAnnotation.FindStringSubmatch(line) + if m == nil || m[1] != want[0] || m[2] != want[1] { + t.Errorf("reader missed the annotation in %q", line) + } + } + for _, line := range []string{ + " RETRIEVAL_AUTH_FAILURE_REASON_UNSPECIFIED = 0; // unset — rejected at ingest", + " // 'expired' is what the edge answers", + " DENIAL_REASON_RATE_LIMITED = 3; // 'rate_limited'", + } { + if tokenAnnotation.MatchString(line) { + t.Errorf("reader matched a line that records no retrieval token: %q", line) + } + } +} diff --git a/docs/design-history.md b/docs/design-history.md index 7740fc13..f8f6c60a 100644 --- a/docs/design-history.md +++ b/docs/design-history.md @@ -1536,6 +1536,48 @@ the deny-by-default scheme allowlist — are **corpus-locked** to shared vectors the residual wiring (that the guard is actually applied on each redirect hop and that a non-2xx surfaces as a status rather than a crash) is covered behaviorally per language. +## Which transport a resolver defaults to is decided by its URL's provenance + +The rule behind the sentence above, stated because the ports mirrored the options struct +that carries it without carrying the rule itself. + +A resolver's nil-default follows where its URL COMES FROM, not what it fetches. A FIXED, +operator-chosen address — the well-known JWKS — defaults to the plain client: the +operator rather than an attacker chose it, and an on-prem directory may legitimately be +private, so guarding it would refuse a deployment its own operator configured. A +REQUEST-DERIVED host — the WBA directory named in a `Signature-Agent` header, an Exchange +domain read off an offer, the Exchange domain on a `RegisterRequest` — defaults to the +guarded one, because the party choosing the address is not the party running the process. +Go states this in `WellKnownOptions.HTTP`'s own comment, one field above both +constructors that read it. + +What went wrong is instructive, and it is the same shape as the WBA scheme gap recorded +above. The ports copied the options struct, the two faces and the field names, and took +the plain default for all of them — then explained it in a docstring as *matching the Go +oracle, which guards only its WBA client*, which had stopped being true when Go's endpoint +resolver was guarded. A wrong sentence in a docstring is load-bearing in a way a wrong +sentence in prose is not: it is the thing the next port reads. The registration-requirements +reader inherited both the default and the sentence, and it was the first component either +port's CLIENT builds on its own — the endpoint resolver has no default there at all, so +until then an application always chose that transport itself. + +Two consequences worth keeping. A guarded default has to be built ONCE per holder, because +in every language the guarded factory constructs a dispatcher or a connection pool; a +`?? guardedFetchFromEnv()` written at a call site trades an unguarded dial for a leaked +pool per call. And the escape stays what it always was: inject a transport, or set +`SKIP_SSRF` / `ALLOW_INSECURE`. Neither of those is a reason to default to the plain client +— they are the reason defaulting to the guarded one costs nothing. + +The ports' endpoint resolver still defaults plain, which is the last leg not following the +rule and is tracked separately. + +Each language now pins its own row of that table in a test that dials a loopback origin +with no transport injected, so moving a default fails a gate rather than merely diverging +from this page and from the published threat model — which states the same postures per +language, and which drifted from the code the first time it was written down. Two rows +deliberately record the gap above, and say so, because a test that pinned an insecure +default without explaining itself would read as endorsement of it. + ## Active-key selection: deterministic, non-normative, unbounded by default The WBA directory can carry several simultaneously window-active signing keys during an @@ -1565,7 +1607,9 @@ Python and TS ship symmetric **decoders** (`parse_error_detail` / `error_detail_ reason enum a Go exchange emitted, never on a human string. Emit and decode are pinned to one shared oracle corpus (`error-detail-vectors.json`) replayed by all three languages, so the typed-failure contract is verified end-to-end across the language -boundary rather than trusted to match by inspection. +boundary rather than trusted to match by inspection. The details a client SYNTHESIZES — +where no peer wrote the values and all three have to author the same ones — are pinned +separately and for a different reason; see "Details the SDK writes itself are pinned". ## Discovery answers are grouped per URI, not flattened @@ -1646,10 +1690,167 @@ checks** stand between the caller's message and the wire, in this order. follows is bounded and evicts least-recently-used, because which Exchanges appear is driven by incoming offers — an open-ended, caller-influenced key space. -## Three client defaults where the ports differ from Go on purpose +## The account verbs route like a report, not like a discovery + +`Register` and `GetAccountStatus` are the agent client's, not a fourth constructor's, and +they read their destination off the message rather than from the base URL. Both halves of +that follow one rule already recorded above: a client's constructor takes its shape from +where the address comes from. + +The party is the same. An agent registering holds the same key it discovers and buys with +— the protocol carries one agent identity — so there is no second signer and no second +address to hang a constructor on. What differs from discovery is *which* Exchange: an +account is per-Exchange, and which Exchange is the agent's choice PER CALL rather than the +operator's at start-up. A registration target routinely arrives at runtime, because a +denial names where to register precisely so an agent can converge unattended. A client +that could only register where it was configured could not follow that. + +So both verbs take the usage report's shape: `exchange` off the request, the endpoint +resolved from that Exchange's own manifest, the guarded leg, and no parameter a configured +origin could be passed as. An earlier design document said the opposite — that these two +calls use the configured address because an account's address is known in advance — and it +was written before accounts were per-Exchange. The contract disagrees with it in as many +words: `RegisterRequest.exchange` says the caller reaches this endpoint by resolving a +fetched manifest. + +Two details the routing tier does not supply, and this leg therefore does itself. The +contract's SHAPE rule runs on the recipient as well as the routing tier's dialability +question: the two are deliberately different predicates, and nothing upstream of the client +has applied the first, so a value the wire would refuse is declined here rather than +carried to an Exchange that can only refuse it. And neither verb takes call options, +because neither message carries an idempotency key — registering again returns the same +account handle, so a key would be ceremony rather than a guarantee. + +## A digest that may not be cached needs a reader that cannot cache + +Submitting a registration states which terms the operator accepted, and the contract +requires the value to come from a FRESHLY fetched manifest: a cached endpoint is fine, a +cached digest is not, because a client cannot detect staleness locally and a warm cache +would make it echo a value the Exchange has already stopped accepting and retry the same +refusal until the cache expired. + +The SDK had one manifest path, and it is built out of exactly the mechanism that value may +not touch — a per-host cache with a TTL, single-flight coalescing on top, and no bypass. +Adding a member to the document it decodes would have satisfied the rule's letter at the +one point it fails in practice: the first caller to reuse the SDK's own manifest path would +be handed a digest minutes old with nothing in the API to warn them. So the requirements +read is a separate face that holds no document cache. Leaving no cache slot to reuse is +what makes the rule structural rather than a convention callers remember — the same +argument the report leg makes about configuration. + +Reading the published `data_schema` out of the same fresh bytes then costs nothing extra +and removes a failure mode of its own: a stale local schema cannot refuse a payload the +Exchange would have taken, because there is no stale local schema. Not even the compiled +validator is held. Memoising it is a cache, which this tier does not do by default, and the +useful key for one — the digest of the served bytes — is a property of a deployment's +threat model rather than of the protocol. + +Two things the ports had to build that Go got for free, both from the same sentence: the +caps are defined over the bytes AS SERVED. Neither JavaScript nor Python has a raw-message +slice, and re-serialising the parsed member measures a document nobody sent — a schema +padded past the cap on the wire that minifies under it would then be refused by an Exchange +and accepted by a client, which is the two-privately-chosen-limits failure those numbers +exist to prevent. Both ports slice the member out of the served body by extent instead. + +The pre-check refuses a payload a USABLE schema rejects, and refuses nothing otherwise. A +schema this SDK will not compile is skipped, because the contract says so and gives the +reason: refusing locally and declining to send would turn a rule about reading a third +party's document into a denial of service against the client's own user. Absent schema and +unusable schema are therefore one branch, not two. + +## A seam that invites a stricter reader needs a word for that reader's refusals + +The requirements read is an injectable seam, offered deliberately: an application that +wants an allow overlay, or a compiled-schema cache this tier will not hold, supplies its +own reader. What the seam did not offer was a way for that reader to say a refusal was +final. It classified on the three sentinels the SDK's own reader raises and read +everything else as a transport failure — so a reader stricter than the SDK's, which is +the only kind anyone writes their own for, had its permanent refusals reported as +transient and retried against a third party's origin indefinitely. + +The first external implementation of the seam found it. That reader validates the whole +manifest — the document must be JSON, must satisfy the manifest's own schema, must carry +a version it recognises, must pass the protocol's field rules — and every one of those +refusals is a property of the bytes that were served. None of them is in the SDK's three, +so all of them arrived as "try again", forever. + +This is the shape recorded twice already in this file: a classification that changes with +the resolver is not a classification, and a verdict the contract does not state gets +inferred from whatever implementation the reader has at hand. `ErrManifestUnusable` states +it. The document arrived, this reader cannot use it, and the next fetch returns the same +bytes. + +**The SDK's own reader never raises it, and that is the point rather than an omission.** +The two ways a manifest disappoints that reader are both deliberate non-errors, and each +is load-bearing somewhere else. An optional member carrying a type the contract does not +admit reads as ABSENT, because the projection is shared with the endpoint and key faces +and one off-spec member must not fail a document those two would have read fine. A +document that does not decode at all is a transport failure, because a proxy serving an +error page under a 200 may well not be serving one on the next attempt, and the leg that +shares this classification carries usage reports, where a wrongly-permanent refusal loses +money. Both stay. The sentinel exists for the reader the SDK did not write. + +**The endpoint seam's sentinels were not reused, and no sentinel was shared between the +two seams.** Their vocabularies are disjoint on purpose: one answers "can this endpoint be +dialled", the other "can this document be read for what a registration owes", and the +verdicts of each are properties of its own question. The single sentinel both seams honour +is the invalid-host refusal, and only because it is the L1 host predicate both +implementations run rather than a verdict either of them reaches. Widening the +requirements seam to the endpoint seam's version refusal would have been the first +crossing, and it would have said a manifest version is a registration concern, which it is +not — a reader that refuses one now has a verdict of its own shape to wrap. + +**One cost, named because it is real and recurring.** A sentinel set is an allowlist, and +an implementation that does not know a member exists gets its answer misread — which has +now happened three times on this seam and its sibling. The alternative shape already ships +one tier over: the content fetch returns a classified error whose finality the caller +reads off a field, so an implementation declares its verdict instead of guessing which +sentinel to wrap. Moving these two seams to that shape is the change that would close this +rather than document it. + +## The peer's token is ours to trust; the peer's sentence is only ours to carry + +A caller reading why a call failed had the failure class and the peer's refusal token, both +machine-readable, and no way to reach the peer's own sentence except by parsing it back out +of a rendered error. That is a thing a layer will get wrong, and the platform's MCP adapter +had grown parallel extraction arms over three error shapes to avoid doing it. + +So the client's failure carries the sentence as a value, beside the token rather than in +it. `Reason` is the peer's machine token; putting prose there was a mistake this SDK made +once and reverted, and the same mistake in the other direction would be no better. + +What it holds is the TYPED reason's developer message and nothing else. It is not filled +from a detail this SDK BUILT ITSELF either — the content leg's sentence around an edge's +token, or the registration pre-check's around schema failures. Those carry a typed reason, +so the rule below about untyped answers would not have stopped them; what stops them is +that the words are ours, and a field that exists so a layer can attribute prose to a remote +party cannot sometimes hold our own. An answer that did +not come from a RAMP service — a draining load balancer, a proxy's own page — carries no +message of its own, and the text a transport synthesizes for one is that transport's: +connect-go writes a status line where a fetch-based client writes nothing. Filling the +field from that would make its value a property of the language rather than of the answer, +which is the drift a shared corpus exists to catch and could not have caught here, because +each language would have been faithfully reporting its own transport. The synthesized text +stays reachable through the cause, where it reads as what it is. + +It is unbounded, and that is a choice rather than an oversight. The contract calls the +field it comes from an easy existence oracle and places the no-secrets duty on the server, +so truncating it is a decision that belongs to whoever displays it, where the audience is +known — the same reason the typed detail's message has never been bounded here either. + +One hazard is documented rather than fixed on this path. `GetAccountStatusRequest` carries +no varying field, so two calls to one Exchange inside a second sign identical bytes and a +peer screening replays on (key id, signature) refuses the second. The remedy is a monotonic +freshness window, which every language ships — but a window is ONE INSTANCE PER CLIENT +rather than one per call, since the running maximum is the whole mechanism, so the choice +belongs to whoever builds the client and no verb makes it for them. Bounding that window's +drift is a separate change: today it compounds without a ceiling, which is its own defect +and has its own answer. + +## Four client defaults where the ports differ from Go on purpose Go is the oracle and the ports mirror it, so a difference that survives review has to be -written down or it reads as drift. Three survive, and none is an API-surface divergence — +written down or it reads as drift. Four survive, and none is an API-surface divergence — all are behaviour, so they live here rather than in `sdk/parity/symbol-map.json`, whose allowlist is a shrink-only ratchet over SYMBOLS. @@ -1698,6 +1899,17 @@ it. Both readers of one wire is the shape this file exists to record, and moving into the contract — which would make it one rule for every implementation rather than two — is the change that would close it rather than document it. +**The default signature validity window is five minutes in Go and TypeScript, and ten in +Python.** It has been that way since the ports were written and was recorded nowhere, +which is how it reads as drift rather than as a decision. It is a genuine difference and +not a rounding: a deployment that cares about the number sets it explicitly, and every +client now names the knob at the same tier so it can — `WithSignWindow` in Go, +`signWindow` in TypeScript, `ClientConfig.sign_window` in Python. What it costs, stated +plainly: a peer enforcing a maximum signature lifetime may accept one client's default +and refuse another's, and the two would then disagree about a request neither client +built wrongly. Moving the number into the contract is the change that would close it +rather than document it. + **The client's transport splits in two, and which leg is guarded is the same in all three.** A plain transport carries the configured home Exchange and the Broker — an operator that points the SDK at a private origin chose that address — and an @@ -1917,3 +2129,70 @@ The claim is scoped to that seam deliberately. A caller that injects a whole ownership of that fetch and the guard is that caller's to install — which is a different bargain from an option that quietly weakens a fetch the SDK still performs. + +## A refusal the client computed carries the same answer the Exchange would have sent + +A registering client pre-checks its payload against the schema the Exchange publishes, so +a payload that will be refused is refused locally, before anything is signed. That check +produces the same structured list of offending members the Exchange produces from the same +validator — and the client flattened it into a sentence and dropped the list. + +The consequence is a layer that has to parse members back out of prose, which is precisely +what the failure contract says a layer must never be asked to do. A consumer rendering +both sources through one renderer, so an agent gets the same remedy whichever side +declined, could not: it had a typed detail from the Exchange and a string from the client, +and it would have read them as two different problems. The refusal now carries the detail +beside the sentence, and the sentence stays because it is what a caller sees who never +opens the detail at all. + +Its DOMAIN names the client's own tier. The Exchange never saw the request, so attributing +the verdict to it would be the same conflation the peer-message rule already refuses — +that field exists so a consumer can attribute words to a remote party, and a domain works +the same way for a surface. The value the field may carry is deliberately open in the +contract: no rule, and an EXAMPLE rather than a set, which is right for a grouping key +that would otherwise have to move every time a tier does. What the values in use actually +obey is narrower, and worth stating because nothing had: a `Service` suffix names an RPC +service the contract DEFINES, and a bare noun names a tier that is not one and appears in +no descriptor — a delivery edge, or the client itself. The suffix is what tells a reader +which they have. A conformance guard now holds every committed domain to that, and found a +vector naming a registration service the contract has never defined. + +The field errors travel as PATHS wherever the three languages have to agree. Each pointer +is a projection of the payload, which all three compute identically; the constraint text +beside it comes from a different JSON Schema library in each, and the contract says so of +that field — validator-defined, not stable across implementations. Pinning the prose would +pin an accident of which library each port happens to use. + +One rendering bug came out of the same place. An empty pointer addresses the whole object, +which is how a missing required member and every other whole-object failure is reported, +and the renderer wrote `path + ": " + error` unconditionally — so the most ordinary +registration failure there is rendered as a member with no name. + +## Details the SDK writes itself are pinned, not just the ones it reads + +Decoding a peer's `ErrorDetail` was already held to one corpus in all three languages. +Two details are not decoded at all. The content leg promotes a delivery edge's refusal +token into a typed reason and writes the envelope around it, because an edge answers a +small JSON object rather than a protobuf. The registration pre-check builds one from +failures no peer ever saw. In both, the failing surface, the sentence and the reason are +AUTHORED — three times, in three languages, from three copies of one decision — and +nothing compared them. + +That is the shape that drifts without anything going red, and it had. The token table +under the edge leg is not derivable from the enum's own spelling: `expired` is +`URL_EXPIRED` and `pop_expired` is `PROOF_EXPIRED`, which is why the proto records the +token beside each value. Two of the three SDKs never read that record. They computed each +reason's name by uppercasing the token and prefixing it, which lands on the recorded value +for two of the eleven tokens and on nothing for the other nine — so nine refusals a real +edge emits carried a typed reason in one language and none in the other two, and the +inverse held too: any string whose uppercase happened to spell an enum suffix was +promoted, including spellings no edge sends. Every suite was green, because none of them +compared a language's answer to the record. + +Both details are now one corpus, driven from the real client rather than described, and +every recorded token is a row — including the two that must stay UNTYPED. The protocol +records one token against TWO values, one per checker, so the body cannot say which ran +and no language may choose; and a spelling no edge emits is recorded as untyped precisely +so a port that derives names from tokens cannot come back green. A second conformance +guard reads the annotations out of the `.proto` source and holds the corpus to them, so +the record rather than any one language is what the three are measured against. diff --git a/docs/sdk-parity-matrix.md b/docs/sdk-parity-matrix.md index 7e607813..810ed650 100644 --- a/docs/sdk-parity-matrix.md +++ b/docs/sdk-parity-matrix.md @@ -12,7 +12,7 @@ Go is the oracle (`sdk/go/{helpers,resolvers,core,connect,connectserver}`); Python and TS mirror it. This document is **generated** from the same two artifacts CI already enforces against the code, so it cannot drift from the real surface — a mismatch fails the API-surface gate or the corpus-completeness gate before it can reach this file. -**At a glance:** 138 symbols at cross-language parity · 16 documented divergences · 183 Go-idiomatic exclusions · 36 conformance corpora, each tri-replayed. +**At a glance:** 144 symbols at cross-language parity · 17 documented divergences · 184 Go-idiomatic exclusions · 37 conformance corpora, each tri-replayed. Layering (L1 pure trust core vs L2 I/O resolvers), the SSRF transport-wiring invariant, and naming conventions are recorded in [`design-history.md`](./design-history.md). @@ -129,20 +129,25 @@ Legend: a name = the public face in that language · `—` = intentionally none | `DefaultMaxContentBytes` | `DEFAULT_MAX_CONTENT_BYTES` | `DEFAULT_MAX_CONTENT_BYTES` | | `ErrDirectoryUnavailable` | `DirectoryUnavailableError` | `DirectoryUnavailable` | | `ErrEndpointRefused` | `EndpointRefusedError` | `EndpointRefused` | +| `ErrExchangeNotPermitted` | `ExchangeNotPermittedError` | `ExchangeNotPermitted` | | `ErrKeyExpired` | `KeyExpiredError` | `KeyExpired` | | `ErrKeyRevoked` | `KeyRevokedError` | `KeyRevoked` | +| `ErrManifestNotExchange` | `ManifestNotExchangeError` | `ManifestNotExchange` | +| `ErrManifestUnusable` | `ManifestUnusableError` | `ManifestUnusable` | | `ErrManifestVersionRefused` | `ManifestVersionRefusedError` | `ManifestVersionRefused` | | `ErrNoEndpoint` | `NoEndpointError` | `NoEndpoint` | | `ErrRevocationUnevaluated` | `RevocationUnevaluatedError` | `RevocationUnevaluated` | | `ErrUnknownKey` | `UnknownKeyError` | `UnknownKey` | | `NewGuardedClientFromEnv` | `guarded_client` | `guardedFetchFromEnv` | | `OfferDirectoryFetcher` | `DirectoryFetch` | `OfferDirectoryFetch` | +| `RegistrationRequirements` | `RegistrationRequirements` | `RegistrationRequirements` | | `SSRFGuard` | `ssrf_guard` | `ssrfGuard` | | `WBADirectoryPath` | `WBA_DIRECTORY_PATH` | `WBA_DIRECTORY_PATH` | | `WBADirectoryURL` | `wba_directory_url` | `wbaDirectoryURL` | | `WBAKeyResolver` | `WBAKeyResolver` | `WBAKeyResolver` | | `WellKnownEndpointResolver` | `WellKnownEndpointResolver` | `WellKnownEndpointResolver` | | `WellKnownKeyResolver` | `WellKnownKeyResolver` | `WellKnownKeyResolver` | +| `WellKnownRequirementsReader` | `WellKnownRequirementsReader` | `WellKnownRequirementsReader` | ### core — L2 composition (verifier, signing transport, windows, replay) @@ -175,6 +180,7 @@ Legend: a name = the public face in that language · `—` = intentionally none | `DefaultMaxRPCReadBytes` | `DEFAULT_MAX_RPC_READ_BYTES` | `DEFAULT_MAX_RPC_READ_BYTES` | | `EndpointResolver` | `EndpointResolver` | `EndpointResolver` | | `ErrorDetailFrom` | `error_detail_from` | `errorDetailFrom` | +| `RegistrationRequirementsReader` | `RegistrationRequirementsReader` | `RegistrationRequirementsReader` | | `Validation` | `Validation` | `Validation` | ### connectserver — server-verify handler binding @@ -209,6 +215,7 @@ Deliberate, reason-backed asymmetries. The allowlist is **shrink-only** — a ne | `resolvers.NewWBAKeyResolver` | — | `createWBAKeyResolver` | Go NewWBAKeyResolver factory folds into the Python class constructor (WBAKeyResolver(...)); idiomatic Python exposes the class, not a separate factory symbol. TS keeps the createWBAKeyResolver factory. | | `resolvers.NewWellKnownEndpointResolver` | — | `createWellKnownEndpointResolver` | Go NewWellKnownEndpointResolver factory folds into the Python class constructor (WellKnownEndpointResolver(...)); idiomatic Python exposes the class, not a separate factory symbol. TS keeps the createWellKnownEndpointResolver factory. | | `resolvers.NewWellKnownKeyResolver` | — | `createWellKnownKeyResolver` | Go NewWellKnownKeyResolver factory folds into the Python class constructor (WellKnownKeyResolver(...)); idiomatic Python exposes the class, not a separate factory symbol. TS keeps the createWellKnownKeyResolver factory. | +| `resolvers.NewWellKnownRequirementsReader` | — | `createWellKnownRequirementsReader` | Go NewWellKnownRequirementsReader factory folds into the Python class constructor (WellKnownRequirementsReader(...)); idiomatic Python exposes the class, not a separate factory symbol. TS keeps the createWellKnownRequirementsReader factory. | | `resolvers.WBAKeyResolverOptions` | — | `WBAKeyResolverOptions` | Go options struct / TS options object folds into Python constructor keyword arguments (WBAKeyResolver(...)); idiomatic Python has no separate options type. | | `resolvers.WellKnownOptions` | — | `WellKnownOptions` | Go options struct (shared by the well-known key/endpoint resolvers) / TS options object folds into Python constructor keyword arguments (WellKnownKeyResolver(...) / WellKnownEndpointResolver(...)); idiomatic Python has no separate options type. | @@ -243,6 +250,7 @@ Go constructs (functional-option builders, `errors.Is` sentinels, value types, c | `connect.WithMaxContentBytes` | Go functional-option builder; py/ts pass options via kwargs/options objects. | | `connect.WithOfferKey` | Go functional-option builder; py/ts pass options via kwargs/options objects. | | `connect.WithProofWindow` | Go functional-option builder; py/ts pass options via kwargs/options objects. | +| `connect.WithRegistrationRequirements` | Go functional-option builder; py/ts pass options via kwargs/options objects. | | `connect.WithRequestIDFunc` | Go functional-option builder; py/ts pass options via kwargs/options objects. | | `connect.WithRequester` | Go functional-option builder; py/ts pass options via kwargs/options objects. | | `connect.WithSignWindow` | Go functional-option builder; py/ts pass options via kwargs/options objects. | @@ -353,7 +361,7 @@ Go constructs (functional-option builders, `errors.Is` sentinels, value types, c | `helpers.RegistrationDataUncanonicalizable` | Member of the mapped helpers.RegistrationDataVerdict vocabulary. Python and TypeScript spell it as a literal; the shared registration-schema corpus pins the token. | | `helpers.RegistrationSchemaCompileTimeout` | Go-only wall-clock backstop on compilation, not part of the accepted/refused contract and deliberately absent from the ports: Go's runtime preempts, while a CPU-bound spin holds CPython's interpreter and blocks Node's event loop, so a timer there cannot interrupt the work it names. What bounds all three identically is static — the size, depth and evaluation caps and the pattern alphabet — and no admitted schema should ever reach this timeout. | | `helpers.RequestAcceptancePayload` | Go typed-protobuf builder for the request-acceptance payload; Python and TypeScript build their language-native payload objects inside the mapped canonicalizer/client. | -| `helpers.RetrievalAuthFailureReasonFromToken` | Go lookup from the delivery edge's refusal token to the typed enum; py/ts branch on the token string directly. | +| `helpers.RetrievalAuthFailureReasonFromToken` | Go exposes the delivery edge's refusal-token table as an L1 lookup, because the Exchange and the client both read it; in the ports the content leg is its only consumer, so each keeps the SAME table private to that module. The table is the protocol's own record — ramp.proto names the token beside each enum value — never derived from the enum's spelling, and synthesized-detail-vectors.json replays every recorded token in all three languages so the three copies cannot come apart again. | | `helpers.SchemaAccepted` | Member of the mapped helpers.SchemaVerdict vocabulary. Python and TypeScript spell it as a literal; the shared registration-schema corpus pins the token. | | `helpers.SchemaCompileTimeout` | Member of the mapped helpers.SchemaVerdict vocabulary. Python and TypeScript spell it as a literal; the shared registration-schema corpus pins the token. | | `helpers.SchemaMalformed` | Member of the mapped helpers.SchemaVerdict vocabulary. Python and TypeScript spell it as a literal; the shared registration-schema corpus pins the token. | @@ -443,5 +451,6 @@ Go emits each `*-vectors.json` oracle; Python and TS replay it. The completeness | `resolvers/testdata/wba-url-vectors.json` | ✅ | ✅ | ✅ | | `connect/testdata/client-request-vectors.json` | ✅ | ✅ | ✅ | | `connect/testdata/connect-error-vectors.json` | ✅ | ✅ | ✅ | +| `connect/testdata/synthesized-detail-vectors.json` | ✅ | ✅ | ✅ | | `connect/testdata/transport-failure-vectors.json` | ✅ | ✅ | ✅ | diff --git a/proto/CHANGELOG.md b/proto/CHANGELOG.md index d4ff66be..a0c26528 100644 --- a/proto/CHANGELOG.md +++ b/proto/CHANGELOG.md @@ -2,6 +2,99 @@ ## Unreleased +**The SDK serves the account-setup role, in all three languages (no wire change; +conformance-affecting).** `Register` and `GetAccountStatus` land on the agent client — +`connect.Client` in Go, `createClient`'s face in TypeScript, `Client` and its blocking +twin in Python — under the same names, over the same signing transport, redirect +refusal, request-id and validate interceptors and read cap as the other agent verbs. +They sit on that client rather than on a fourth constructor because they are the same +party holding the same key, and they route as a usage report does: the destination is +read off the request's own `exchange`, resolved from that Exchange's own manifest, and +dialled on the guarded leg. An account is per-Exchange and which Exchange is the agent's +choice per call — a denial names where to register — so a configured origin would confine +registration to one Exchange. Neither message carries an idempotency key, so neither verb +takes call options; `ver` is stamped when empty; a request that names no bare-domain +recipient is refused before it is signed. The client-request corpus gains three rows, +replayed in all three languages. + +A new resolver face reads what one Exchange asks of a registration — the terms revision +submitting one accepts, and the schema its `registration_data` must match — from that +Exchange's own manifest, and it holds NO document cache. The contract requires a +registering client to read the terms digest from a freshly fetched manifest, and the +endpoint resolver is built out of exactly the mechanism that value may not touch, so a +face with no cache slot to reuse is what makes the rule structural. `Register` fills +`terms_digest` from it only when the caller left the field unset, applies the four +`registration_data` bounds before anything is signed, and pre-checks the payload against +the published schema — skipping the check, never the send, when that schema is one the +SDK refuses. The schema is measured over the bytes AS SERVED in every language, which the +two JSON ports reach by slicing the member out of the served body rather than +re-serialising a parsed value. + +The reader dials on the SSRF-guarded transport in all three languages, and is built once +with the client rather than once per registration: the Exchange domain comes off the +request, so it is an address another party chose, which is the provenance that takes the +guard. A deployment reaching a private Exchange injects its own transport or sets the +usual two environment flags. + +Two cross-cutting changes anyone re-pinning will see. The client's typed failure now +carries the peer's own developer message as a value — filled where the peer's own answer +is decoded, and empty otherwise: never a transport's synthesized status line, and never a +typed detail the SDK built itself, as the content leg does from an edge's refusal token. +So a consumer reads it instead of parsing it back out of a rendered error; it is +unbounded, and bounding it belongs to whoever displays it. And Python's `ClientConfig` gains `sign_window`, so the +RFC 9421 freshness knob sits at the tier Go and TypeScript already surface it at. + +A refusal the pre-check computes carries what it computed. The offending members travel as +a typed `RegistrationFailure` detail beside the sentence, so a consumer that renders the +Exchange's refusal and the client's own through one renderer reads one shape either way +rather than parsing members back out of prose. Its domain names the client's own tier: the +Exchange never saw the request. The field errors are compared across languages by POINTER +only — the constraint text beside each comes from a different JSON Schema library in each +port and the contract calls it validator-defined — and an empty pointer, which is how a +missing required member is reported, now renders without a leading separator instead of as +a member with no name. + +Two corrections to the content leg travel with it. The delivery edge's refusal tokens are +recorded in the proto beside each `RetrievalAuthFailureReason`, and the mapping is not +derivable — `expired` is `URL_EXPIRED`, `pop_expired` is `PROOF_EXPIRED`. Both JSON ports +had computed each name by uppercasing the token instead, which matches the record for two +of the eleven tokens, so nine refusals a real edge emits reached a TypeScript or Python +caller with no typed reason at all while any spelling that happened to match an enum +suffix was promoted. Both now carry the record. And an invalid-host refusal from an +injected requirements reader is classified `not_sent` in all three rather than retried in +two, which is what each port's own routing leg already answered. + +**The requirements seam gains a verdict for a document it cannot use (additive, no wire +change).** `resolvers.ErrManifestUnusable` / `ManifestUnusableError` / `ManifestUnusable` +says the document arrived and this reader cannot use it — final, not a transport failure, +because the next fetch returns the same bytes. It exists because the reader seam is +injectable: the SDK's own reader refuses three things and treats every other +disappointment as absence or as an outage, so an implementation stricter than it — one +validating the whole manifest, or refusing a version — held a permanent refusal the seam +reported as transient and a caller retried indefinitely. One consequence for anyone +re-pinning: this is a fourth verdict on that seam, and a classifier branching only on the +older three drops it into its transport-failure bucket and retries something that will +never succeed; add the new sentinel alongside. The SDK's own reader never returns it — +an off-spec optional member still reads as absent, and an undecodable document is still a +transport failure — so nothing changes for a caller that injects no reader. Python also +stops treating an unrelated `ValueError` as a verdict on both the routing and the account +leg: the invalid-host refusal is recognised by its wording now, as TypeScript already did, +so an injected seam raising `json.JSONDecodeError` is retryable there as it is in the +other two. + +A new shared corpus pins the details the SDK builds itself rather than receives — the two +above — over every recorded edge token, including the two that must stay untyped, replayed +in all three languages. Two conformance guards sit under it: one holds every committed +`ErrorDetail.domain` to the shape it claims (a `Service` suffix names a service the +contract defines, a bare noun names a tier that is not one), and one holds the corpus to +the token annotations in the `.proto` source. The first found a vector naming a +registration service the contract has never defined; that fixture is corrected. + +*Parity record:* six new mapped symbols and one Go-idiomatic exclusion; the reader's Go +factory folds into the Python class constructor as every other `NewX` does, so the +shrink-only allowlist baseline moves 16 → 17 as a reviewed bump under that one recorded +class. + **SDK endpoint resolvers enforce `WellKnownManifest.ver`; `WellKnownManifestVersion` exported (SDK behaviour change; no wire change).** The manifest's `ver` comment now states the full receive-side rule — read before any other member, accept a diff --git a/sdk/go/README.md b/sdk/go/README.md index c816cb31..6e52217c 100644 --- a/sdk/go/README.md +++ b/sdk/go/README.md @@ -8,8 +8,8 @@ directly with no `replace` directive. |---|---|---| | **L0** | `gen/go/ramp/v1`, `gen/go/vocab/*` | generated wire types (consumed, never rebuilt) | | **L1** | **`sdk/go/helpers`** | stateless, **IO-free** protocol helpers — RFC 9421/7638 crypto, offer/acceptance verify, static key resolution, validation | -| L2 · I/O | **`sdk/go/resolvers`** | the network-fetching tier: well-known JWKS / WBA directory / `ramp.json` endpoint / offer-key resolvers + the SSRF-guarded HTTP client. Runs on a maintained `net/http` client behind the SSRF guard; composes L1, never the reverse | -| L2 · transport | `sdk/go/core` (transport-neutral: Verifier, {verified,rejected}, `DiscoveryResult` per-URI groups, VerifiedOffer guard, signing RoundTripper, ReplayStore — zero Connect) · `sdk/go/connect` (Connect **client** binding: `NewClient` + `NewBrokerClient` + `NewCatalogClient`; the agent verbs **`Discover` · `Resolve` · `Execute` · `ReportUsage` · `Dispute` · `Fetch`** and the publisher verbs **`PushResources` · `RemoveResources` · `RefreshCatalog`** + client options + the `CallError` taxonomy + `ErrorDetailFrom`) · `sdk/go/connectserver` (Connect **server** binding: `NewExchangeServiceHandler` + `NewBrokerServiceHandler` + `NewCatalogServiceHandler` + server options + `AsConnectError` + `AttachErrorDetail`/`AttachDetail` + the reject answer **`RejectCode` · `IsBodyTooLarge` · `WriteReject`**, the one place the 413/429/401 split and the error-envelope body are decided) | transport-neutral core + Connect client/server bindings (state injected) | +| L2 · I/O | **`sdk/go/resolvers`** | the network-fetching tier: well-known JWKS / WBA directory / `ramp.json` endpoint / offer-key resolvers, the uncached registration-requirements reader, + the SSRF-guarded HTTP client. Runs on a maintained `net/http` client behind the SSRF guard; composes L1, never the reverse | +| L2 · transport | `sdk/go/core` (transport-neutral: Verifier, {verified,rejected}, `DiscoveryResult` per-URI groups, VerifiedOffer guard, signing RoundTripper, ReplayStore — zero Connect) · `sdk/go/connect` (Connect **client** binding: `NewClient` + `NewBrokerClient` + `NewCatalogClient`; the agent verbs **`Discover` · `Resolve` · `Execute` · `ReportUsage` · `Dispute` · `Fetch`**, the account-setup verbs **`Register` · `GetAccountStatus`** and the publisher verbs **`PushResources` · `RemoveResources` · `RefreshCatalog`** + client options + the `CallError` taxonomy + `ErrorDetailFrom`) · `sdk/go/connectserver` (Connect **server** binding: `NewExchangeServiceHandler` + `NewBrokerServiceHandler` + `NewCatalogServiceHandler` + server options + `AsConnectError` + `AttachErrorDetail`/`AttachDetail` + the reject answer **`RejectCode` · `IsBodyTooLarge` · `WriteReject`**, the one place the 413/429/401 split and the error-envelope body are decided) | transport-neutral core + Connect client/server bindings (state injected) | | L3 | separate packages | framework adapters (convert, never replace) — later | The `L2` tier is split by kind: the **I/O** package (`resolvers`) is the only tier @@ -198,7 +198,16 @@ whole object) plus the violated constraint. The text states the constraint and refusal travels back over the wire, so the validating library's own messages, which quote the failing value, are deliberately not used. The list is deduplicated by pointer and keyword and sorted before the 64-item cap, so the same entries survive in -every language. +every language. The POINTERS survive identically; the constraint text does not, and +the contract says so of that field — it is validator-defined and not stable across +implementations, so branch on the pointer and the typed reason, never on the prose. + +The client's own pre-check returns this same list, on the `CallError` it refuses with, +as a `RegistrationFailure` detail reachable through `ErrorDetailFrom`. That is +deliberate: an agent developer's tooling renders the Exchange's refusal and the local +one through the same code, and a structured list on one side and a sentence on the +other would read as two different problems. Its `domain` names the client rather than +the Exchange, which never saw the request. **Also:** RFC 7638 `Thumbprint`, ADR-019 `ErrorDetail` constructors + `AsConnectError`/`ErrorDetailFrom`/`Reason`, `NewIdempotencyKey`, scope helpers, @@ -229,6 +238,19 @@ import "github.com/RAMP-Protocol/protocol/sdk/go/resolvers" a transport failure and worth retrying. The key is an offer-supplied host, so the cache evicts least-recently-used at a fixed cap and concurrent lookups for one host coalesce to a single fetch. +- **Registration-requirements reader** — `NewWellKnownRequirementsReader` reads what + one Exchange asks of a registration — the terms revision submitting one accepts and + the schema its `registration_data` must match — from the same + `/.well-known/ramp.json`, and holds NO document cache: the contract requires the + digest to come from a freshly fetched manifest. Three + sentinels, same retry question as above: `ErrExchangeNotPermitted` when the + deployment's own overlay excluded the domain before anything was dialled, + `ErrManifestNotExchange` when the document describes another role, and + `ErrManifestUnusable` — which this reader never raises, and which exists because the + seam is injectable: a reader stricter than this one needs a way to say its refusal is + final. All three are verdicts; anything else is a transport failure and worth + retrying. An off-spec optional member reads as ABSENT rather than failing the + document, because the projection is shared with the two faces above. - **Active-key selection** — `ActiveEd25519Key` / `ActiveEd25519KeyWithExpiry` pick an identity's window-active key by document order; the `…Screened` variants fold in a revoked-thumbprint screen. `NewCachedOfferKeyResolver` caches the selected offer diff --git a/sdk/go/connect/account.go b/sdk/go/connect/account.go new file mode 100644 index 00000000..1c39d5f7 --- /dev/null +++ b/sdk/go/connect/account.go @@ -0,0 +1,233 @@ +package connect + +import ( + "context" + "errors" + "fmt" + + connectrpc "connectrpc.com/connect" + + rampv1 "github.com/RAMP-Protocol/protocol/gen/go/ramp/v1" + "github.com/RAMP-Protocol/protocol/sdk/go/helpers" + "github.com/RAMP-Protocol/protocol/sdk/go/resolvers" +) + +// The account-setup verbs. +// +// An agent becomes KNOWN to an Exchange just by sending a signed request; to +// transact for paid content it must REGISTER, and the Exchange mints the opaque +// per-Exchange account handle it will resolve from the request signature on every +// later call. These two verbs are that role's whole surface. +// +// They route like a usage report, not like discovery. An account is per-Exchange, +// and which Exchange is the agent's choice PER CALL: a target routinely arrives at +// runtime — a denial names where to register — rather than from the operator's own +// configuration. So the destination is read off the request's own `exchange` field +// and resolved through that Exchange's own manifest, over the guarded leg, and +// there is no parameter a configured origin could be passed as. +// +// Neither message carries an idempotency_key, so neither verb takes CallOptions. +// That is the contract's choice rather than an omission: registering again for the +// same agent returns the same account handle, so a key would be ceremony rather +// than a guarantee. + +// clientErrorDomain is the ErrorDetail domain for a refusal THIS CLIENT computed, +// before anything was sent. It names the failing surface, which here is the client's +// own tier: the Exchange never saw the request, so naming it would attribute a local +// verdict to a party that reached none. The naming rule the value follows is recorded +// on edgeErrorDomain, beside the other detail this SDK builds itself. +const clientErrorDomain = "ramp.v1.Client" + +// Register creates the calling agent's account at the Exchange the request names. +// +// The caller's identity is the request SIGNATURE. Nothing in the message says who +// is registering, and the business payload is not an identity claim — so a client +// with no signer can build this request and cannot usefully send it. +// +// Four bounds on registration_data are checked before anything is signed, in the +// order the contract fixes: top-level member count, nesting depth, whether the +// payload has a canonical JSON form at all, and the size of that form. They run +// here because a limit that exists to stop work belongs before the work it would +// stop, and because a payload breaking one is a MALFORMED request rather than a +// schema failure — a distinction the Exchange also makes. +// +// terms_digest is filled only when the caller left it UNSET. Submitting a +// registration states which terms the operator accepted, the request signature +// covers that echo, and the contract requires the value to come from a FRESHLY +// fetched manifest — a cached endpoint is fine, a cached digest is not. So the +// fill reads the Exchange's requirements through the uncached reader, which is +// also where the published data_schema comes from, and the payload is pre-checked +// against it before signing. A caller that sets the field is managing its own +// requirements and gets neither the fill nor the pre-check; proto3 presence is +// what lets "unset" and "deliberately empty" be different requests. +// +// The pre-check never becomes a local veto for a schema this SDK refuses. The +// contract is explicit that refusing locally and declining to send would turn a +// rule about reading a third party's document into a denial of service against +// the caller's own user, so an unusable schema is skipped and the Exchange's own +// enforcement decides. A schema that IS usable and that the payload fails is a +// different case: that is the pre-check working, and the request is refused here +// with the offending members named. +// +// Two manifest reads happen on a first registration — one cached, for the +// endpoint, and one fresh, for the digest. That is the contract's own split, and +// registration happens once per Exchange, so the extra fetch is cheap. +// +// A refused registration comes back as a non-OK call whose typed reason is +// readable through ErrorDetailFrom as a RegistrationFailure. +func (c *Client) Register( + ctx context.Context, req *rampv1.RegisterRequest, +) (*rampv1.RegisterResponse, error) { + const op = "register" + if req == nil { + return nil, malformed(op, errors.New("request is nil")) + } + sent, err := cloneRequest(req, op) + if err != nil { + return nil, err + } + stampVer(&sent.Ver) + if err := requireRecipient(op, sent.GetExchange()); err != nil { + return nil, err + } + if verdict := helpers.CheckRegistrationDataStruct( + sent.GetRegistrationData()); verdict != helpers.RegistrationDataAccepted { + return nil, malformed(op, fmt.Errorf("registration_data: %s", verdict)) + } + if sent.TermsDigest == nil { + if err := c.applyRegistrationRequirements(ctx, op, sent); err != nil { + return nil, err + } + } + endpoint, err := vetExchangeEndpoint(ctx, c.endpoints, sent.GetExchange(), op) + if err != nil { + return nil, err + } + resp, err := c.exchanges.clientFor(endpoint).Register(ctx, connectrpc.NewRequest(sent)) + if err != nil { + return nil, sendError(op, err) + } + return resp.Msg, nil +} + +// GetAccountStatus reports whether the calling agent's account at the named +// Exchange is active. +// +// The request carries no field identifying the caller — the Exchange resolves the +// account from the verified signature — so `exchange` is the only thing that says +// which account is being asked about. Accounts are per-Exchange, and that is not +// derivable from anything else in the message. +// +// An answer carrying an empty account handle is a NORMAL answer: it means this +// agent holds no account at that Exchange yet. The response also reports the terms +// revision the account accepted, which is what was agreed rather than what is +// published now; comparing the two is how an agent discovers that an operator's +// terms moved under an account it already holds. +// +// A caveat worth knowing before calling this in a loop. The request has no varying +// field, so two calls to the same Exchange inside one wall-clock second sign +// IDENTICAL bytes — signature timestamps have one-second resolution — and a peer +// screening replays on (key id, signature) refuses the second as a duplicate. This +// verb does not choose the freshness window for you, because a window is one +// instance per client rather than per call and the choice belongs to whoever built +// the client: pass core.MonotonicWindow through WithSignWindow when repeat calls +// are expected. +func (c *Client) GetAccountStatus( + ctx context.Context, req *rampv1.GetAccountStatusRequest, +) (*rampv1.GetAccountStatusResponse, error) { + const op = "get account status" + if req == nil { + return nil, malformed(op, errors.New("request is nil")) + } + sent, err := cloneRequest(req, op) + if err != nil { + return nil, err + } + stampVer(&sent.Ver) + if err := requireRecipient(op, sent.GetExchange()); err != nil { + return nil, err + } + endpoint, err := vetExchangeEndpoint(ctx, c.endpoints, sent.GetExchange(), op) + if err != nil { + return nil, err + } + resp, err := c.exchanges.clientFor(endpoint).GetAccountStatus(ctx, connectrpc.NewRequest(sent)) + if err != nil { + return nil, sendError(op, err) + } + return resp.Msg, nil +} + +// applyRegistrationRequirements reads what the Exchange asks of a registration and +// applies it to the request being built: the terms digest is echoed, and the +// payload is pre-checked against the published schema. +// +// A failed READ refuses the registration rather than sending without a digest. +// Guessing here is not the cautious option: an Exchange that publishes a digest +// refuses a registration that omits one, so sending anyway trades a local failure +// the caller can act on for a remote one it cannot. +// +// The read's own verdicts are classified the way the routing tier classifies its +// own — a value the Exchange or this deployment refused is final, anything else is +// a transport failure worth retrying — so a caller branches on one taxonomy +// whichever check declined. +func (c *Client) applyRegistrationRequirements( + ctx context.Context, op string, sent *rampv1.RegisterRequest, +) error { + reqs, err := c.requirements.ResolveRegistrationRequirements(ctx, sent.GetExchange()) + if err != nil { + if errors.Is(err, helpers.ErrInvalidHost) || + errors.Is(err, resolvers.ErrExchangeNotPermitted) || + errors.Is(err, resolvers.ErrManifestNotExchange) || + errors.Is(err, resolvers.ErrManifestUnusable) { + return notSent(op, err) + } + return &CallError{Kind: CallUnreachable, Op: op, Err: err} + } + sent.TermsDigest = reqs.TermsDigest + // A nil validator reports no failures, which is the behaviour the contract + // requires both when the Exchange publishes no schema and when it publishes one + // this SDK refused. One branch, deliberately: distinguishing them here would be + // distinguishing two cases the caller must treat the same. + if fails := reqs.Schema.Validate(sent.GetRegistrationData().AsMap()); len(fails) > 0 { + // The failures travel as a typed detail, not only as prose. The Exchange + // attaches this same list when it refuses the same payload, computed by this + // same validator, so a consumer that renders one refusal renders both — and + // asking it to parse the members back out of a sentence is the thing + // PeerMessage exists to say a layer must never be asked to do. + return &CallError{ + Kind: CallMalformed, + Op: op, + Err: fmt.Errorf("registration_data does not match the schema %s publishes: %s", + sent.GetExchange(), renderFieldErrors(fails)), + Detail: helpers.RegistrationFailureDetail( + clientErrorDomain, + "registration_data does not match the published data_schema", + rampv1.RegistrationFailureReason_REGISTRATION_FAILURE_REASON_INVALID_REGISTRATION_DATA, + fails..., + ), + } + } + return nil +} + +// renderFieldErrors names the offending members in a refusal a human reads. The +// same failures also travel as the typed detail on that refusal, and an Exchange +// attaches them the same way; this is the local pre-check's own rendering, and it +// stays a sentence because nothing branches on it. +func renderFieldErrors(fails []*rampv1.RegistrationFieldError) string { + out := "" + for i, f := range fails { + if i > 0 { + out += "; " + } + // An empty path addresses the whole object, which is how a missing required + // member and every other whole-object failure is reported. Rendering a bare + // ": ..." there would read as a member with no name. + if p := f.GetPath(); p != "" { + out += p + ": " + } + out += f.GetError() + } + return out +} diff --git a/sdk/go/connect/account_test.go b/sdk/go/connect/account_test.go new file mode 100644 index 00000000..efcf50db --- /dev/null +++ b/sdk/go/connect/account_test.go @@ -0,0 +1,689 @@ +package connect_test + +import ( + "context" + "encoding/json" + "errors" + "fmt" + "net/http" + "strings" + "testing" + "time" + + connectrpc "connectrpc.com/connect" + "google.golang.org/protobuf/types/known/structpb" + + rampv1 "github.com/RAMP-Protocol/protocol/gen/go/ramp/v1" + "github.com/RAMP-Protocol/protocol/gen/go/ramp/v1/rampv1connect" + rampconnect "github.com/RAMP-Protocol/protocol/sdk/go/connect" + rampserver "github.com/RAMP-Protocol/protocol/sdk/go/connectserver" + "github.com/RAMP-Protocol/protocol/sdk/go/core" + "github.com/RAMP-Protocol/protocol/sdk/go/helpers" + "github.com/RAMP-Protocol/protocol/sdk/go/resolvers" +) + +// --------------------------------------------------------------------------- +// Origin +// --------------------------------------------------------------------------- + +// recordingAccount answers the two account RPCs and keeps what arrived, so a test +// can assert on the message that actually reached the wire rather than on the one +// the caller handed in. +type recordingAccount struct { + rampv1connect.UnimplementedExchangeServiceHandler + + gotRegister *rampv1.RegisterRequest + gotStatus *rampv1.GetAccountStatusRequest + registers int + + billingRef string + active bool + // refuse, when set, is returned instead of an answer, so a test can drive the + // typed-refusal path. + refuse error +} + +func (a *recordingAccount) Register( + _ context.Context, req *connectrpc.Request[rampv1.RegisterRequest], +) (*connectrpc.Response[rampv1.RegisterResponse], error) { + a.gotRegister = req.Msg + a.registers++ + if a.refuse != nil { + return nil, a.refuse + } + return connectrpc.NewResponse(&rampv1.RegisterResponse{ + Ver: helpers.ProtocolVersion, BillingRef: a.billingRef, Active: a.active, + }), nil +} + +func (a *recordingAccount) GetAccountStatus( + _ context.Context, req *connectrpc.Request[rampv1.GetAccountStatusRequest], +) (*connectrpc.Response[rampv1.GetAccountStatusResponse], error) { + a.gotStatus = req.Msg + if a.refuse != nil { + return nil, a.refuse + } + return connectrpc.NewResponse(&rampv1.GetAccountStatusResponse{ + Ver: helpers.ProtocolVersion, BillingRef: a.billingRef, Active: a.active, + }), nil +} + +// registrationData builds the Struct a RegisterRequest carries. +func registrationData(t *testing.T, m map[string]any) *structpb.Struct { + t.Helper() + s, err := structpb.NewStruct(m) + if err != nil { + t.Fatalf("build registration_data: %v", err) + } + return s +} + +const digestAB = "sha256:abababababababababababababababababababababababababababababababab" + +// legalEntitySchema requires one member, so a payload without it fails the +// pre-check and a payload with it passes. +const legalEntitySchema = `{"type":"object","required":["legal_entity"],` + + `"properties":{"legal_entity":{"type":"string"}}}` + +// bothShapesSchema fails two ways at once: a required member is absent, which is a +// whole-object violation the wire reports with an EMPTY path, and a member that IS +// present breaks its pattern, which is reported against its own pointer. One +// refusal carrying both is what a consumer has to render, and the empty path is the +// half that is easiest to render wrong. +const bothShapesSchema = `{"type":"object","required":["legal_entity"],` + + `"properties":{"vat_id":{"type":"string","pattern":"^[A-Z]{2}[0-9]+$"}}}` + +// --------------------------------------------------------------------------- +// Register +// --------------------------------------------------------------------------- + +// The whole first-registration path: the request is signed, `ver` is stamped, the +// caller's recipient survives, and terms_digest is echoed from the manifest the +// Exchange served on THIS call. +func TestRegister_EchoesTheFreshlyFetchedTermsDigest(t *testing.T) { + sig := newSigningFixture(t) + origin := &recordingAccount{billingRef: "acct-1", active: true} + digest := digestAB + domain, _ := selfAdvertisingExchangeWith(t, sig, origin, func() map[string]any { + return map[string]any{"terms_digest": digest} + }) + + client := rampconnect.NewClient("http://home.invalid", + append(allowLoopback(t), rampconnect.WithSigner(sig.signer))...) + + req := &rampv1.RegisterRequest{ + Exchange: domain, + RegistrationData: registrationData(t, map[string]any{"legal_entity": "Acme"}), + } + resp, err := client.Register(context.Background(), req) + if err != nil { + t.Fatalf("register: %v", err) + } + if resp.GetBillingRef() != "acct-1" || !resp.GetActive() { + t.Fatalf("response = %v", resp) + } + if got := origin.gotRegister.GetTermsDigest(); got != digest { + t.Fatalf("terms_digest on the wire = %q, want the manifest's %q", got, digest) + } + if got := origin.gotRegister.GetVer(); got != helpers.ProtocolVersion { + t.Fatalf("ver = %q, want %q", got, helpers.ProtocolVersion) + } + if got := origin.gotRegister.GetExchange(); got != domain { + t.Fatalf("exchange = %q, want the caller's %q", got, domain) + } + // The caller's message crossed a package boundary as an argument, not as a + // buffer to fill in. + if req.TermsDigest != nil { + t.Fatalf("the caller's request was modified: terms_digest = %q", *req.TermsDigest) + } +} + +// A caller that sets the digest is managing its own requirements: its value is +// sent unchanged and no requirements read happens at all. +func TestRegister_ACallerSuppliedDigestSuppressesTheRead(t *testing.T) { + sig := newSigningFixture(t) + origin := &recordingAccount{billingRef: "acct-1"} + domain, wkHits := selfAdvertisingExchangeWith(t, sig, origin, func() map[string]any { + return map[string]any{"terms_digest": digestAB} + }) + + client := rampconnect.NewClient("http://home.invalid", + append(allowLoopback(t), rampconnect.WithSigner(sig.signer))...) + + mine := "sha256:cdcdcdcdcdcdcdcdcdcdcdcdcdcdcdcdcdcdcdcdcdcdcdcdcdcdcdcdcdcdcdcd" + if _, err := client.Register(context.Background(), &rampv1.RegisterRequest{ + Exchange: domain, + TermsDigest: &mine, + }); err != nil { + t.Fatalf("register: %v", err) + } + if got := origin.gotRegister.GetTermsDigest(); got != mine { + t.Fatalf("terms_digest = %q, want the caller's %q", got, mine) + } + // One manifest fetch, for the endpoint. The requirements read did not happen. + if n := wkHits.Load(); n != 1 { + t.Fatalf("manifest fetches = %d, want 1 (endpoint only)", n) + } +} + +// An Exchange publishing no digest is the ordinary pass-through case: the field +// stays absent rather than becoming an empty string, which is a different request. +func TestRegister_NoPublishedDigestLeavesTheFieldAbsent(t *testing.T) { + sig := newSigningFixture(t) + origin := &recordingAccount{billingRef: "acct-1"} + domain, _ := selfAdvertisingExchange(t, sig, origin) + + client := rampconnect.NewClient("http://home.invalid", + append(allowLoopback(t), rampconnect.WithSigner(sig.signer))...) + + if _, err := client.Register(context.Background(), &rampv1.RegisterRequest{ + Exchange: domain, + }); err != nil { + t.Fatalf("register: %v", err) + } + if origin.gotRegister.TermsDigest != nil { + t.Fatalf("terms_digest = %q, want absent", *origin.gotRegister.TermsDigest) + } +} + +// The published schema is the client's pre-check, and it runs before signing: a +// payload the Exchange would refuse never reaches the wire. +func TestRegister_PreChecksThePublishedSchema(t *testing.T) { + sig := newSigningFixture(t) + origin := &recordingAccount{billingRef: "acct-1"} + domain, _ := selfAdvertisingExchangeWith(t, sig, origin, func() map[string]any { + return map[string]any{ + "account_registration": map[string]any{ + "data_schema": json.RawMessage(legalEntitySchema), + }, + } + }) + + client := rampconnect.NewClient("http://home.invalid", + append(allowLoopback(t), rampconnect.WithSigner(sig.signer))...) + + _, err := client.Register(context.Background(), &rampv1.RegisterRequest{ + Exchange: domain, + RegistrationData: registrationData(t, map[string]any{"trading_name": "Acme"}), + }) + var cerr *rampconnect.CallError + if !errors.As(err, &cerr) || cerr.Kind != rampconnect.CallMalformed { + t.Fatalf("err = %v, want a malformed CallError", err) + } + if !strings.Contains(cerr.Error(), "legal_entity") { + t.Fatalf("refusal does not name the offending member: %v", cerr) + } + if origin.gotRegister != nil { + t.Fatal("a payload refused by the local pre-check reached the wire") + } + + // The conforming payload goes through against the same schema. + if _, err := client.Register(context.Background(), &rampv1.RegisterRequest{ + Exchange: domain, + RegistrationData: registrationData(t, map[string]any{"legal_entity": "Acme"}), + }); err != nil { + t.Fatalf("conforming payload refused: %v", err) + } +} + +// refusedByTheSchema drives a pre-check refusal against schema, so the tests below +// share one arrangement and differ only in what they read off the failure. +func refusedByTheSchema( + t *testing.T, schema string, payload map[string]any, +) (*rampconnect.CallError, *recordingAccount) { + t.Helper() + sig := newSigningFixture(t) + origin := &recordingAccount{billingRef: "acct-1"} + domain, _ := selfAdvertisingExchangeWith(t, sig, origin, func() map[string]any { + return map[string]any{ + "account_registration": map[string]any{ + "data_schema": json.RawMessage(schema), + }, + } + }) + + client := rampconnect.NewClient("http://home.invalid", + append(allowLoopback(t), rampconnect.WithSigner(sig.signer))...) + _, err := client.Register(context.Background(), &rampv1.RegisterRequest{ + Exchange: domain, + RegistrationData: registrationData(t, payload), + }) + + var cerr *rampconnect.CallError + if !errors.As(err, &cerr) || cerr.Kind != rampconnect.CallMalformed { + t.Fatalf("err = %v, want a malformed CallError", err) + } + if origin.gotRegister != nil { + t.Fatal("a payload refused by the local pre-check reached the wire") + } + return cerr, origin +} + +// What the pre-check computed is what the caller gets. An Exchange attaches this +// same list, from this same validator, when it refuses the same payload — so a +// consumer that renders one refusal renders both, and nothing has to parse the +// members back out of a sentence. +func TestRegister_PreCheckRefusalCarriesItsFieldErrors(t *testing.T) { + cerr, _ := refusedByTheSchema(t, bothShapesSchema, map[string]any{"vat_id": "de1"}) + + detail, ok := rampconnect.ErrorDetailFrom(cerr) + if !ok { + t.Fatalf("the pre-check refusal carries no typed detail: %v", cerr) + } + if got := detail.GetRegistrationFailure().GetReason(); got != + rampv1.RegistrationFailureReason_REGISTRATION_FAILURE_REASON_INVALID_REGISTRATION_DATA { + t.Fatalf("reason = %v, want INVALID_REGISTRATION_DATA", got) + } + + // The PATHS are what this asserts, never the constraint prose beside them: that + // text comes from each language's validator library, and the contract calls it + // non-authoritative for exactly that reason. + var paths []string + for _, f := range detail.GetRegistrationFailure().GetFieldErrors() { + paths = append(paths, f.GetPath()) + } + if got, want := strings.Join(paths, ","), ",/vat_id"; got != want { + t.Fatalf("field-error paths = %q, want %q", got, want) + } +} + +// An empty path addresses the whole object, which is how a missing required member +// is reported. Rendering a bare ": ..." there would read as a member with no name. +func TestRegister_ARootViolationRendersWithNoMemberName(t *testing.T) { + cerr, _ := refusedByTheSchema(t, bothShapesSchema, map[string]any{"vat_id": "de1"}) + + msg := cerr.Error() + if strings.Contains(msg, "publishes: : ") { + t.Fatalf("a whole-object violation rendered as a member with no name: %v", msg) + } + // The member that HAS a name still carries it, so the fix did not drop the + // separator everywhere. + if !strings.Contains(msg, "/vat_id: ") { + t.Fatalf("refusal does not address the offending member: %v", msg) + } +} + +// refusingRequirements is a reader that declines every read with one error, so the +// classification of that error is the only thing the test varies. +type refusingRequirements struct{ err error } + +func (r refusingRequirements) ResolveRegistrationRequirements( + _ context.Context, _ string, +) (resolvers.RegistrationRequirements, error) { + return resolvers.RegistrationRequirements{}, r.err +} + +// A failed READ is classified the way the routing tier classifies its own: a value +// this deployment or the Exchange refused is FINAL, anything else is a transport +// failure worth retrying. Without the split a caller retries a refusal forever. +// +// All four verdicts are reachable only through an INJECTED reader — the verb's own +// recipient check runs the host rule first, the SDK's reader applies the next two +// itself, and it never returns the fourth at all — which is exactly why they need a +// test: nothing else exercises them, and a consumer that injects a reader is the +// case this classification exists for. +func TestRegister_ClassifiesARefusedRequirementsRead(t *testing.T) { + for _, tc := range []struct { + name string + err error + want rampconnect.CallErrorKind + }{ + {"not a host", fmt.Errorf("%w: %q is not a bare domain", + helpers.ErrInvalidHost, "exchange.test/path"), rampconnect.CallNotSent}, + {"this deployment refused the host", fmt.Errorf("%w: blocked", + resolvers.ErrExchangeNotPermitted), rampconnect.CallNotSent}, + {"the document is not an Exchange's", fmt.Errorf("%w: wrong role", + resolvers.ErrManifestNotExchange), rampconnect.CallNotSent}, + {"a stricter reader cannot use the document", fmt.Errorf("%w: ver 2.0", + resolvers.ErrManifestUnusable), rampconnect.CallNotSent}, + {"the read never completed", errors.New("connection reset"), + rampconnect.CallUnreachable}, + } { + t.Run(tc.name, func(t *testing.T) { + sig := newSigningFixture(t) + origin := &recordingAccount{billingRef: "acct-1"} + domain, _ := selfAdvertisingExchange(t, sig, origin) + + client := rampconnect.NewClient("http://home.invalid", + append(allowLoopback(t), + rampconnect.WithSigner(sig.signer), + rampconnect.WithRegistrationRequirements(refusingRequirements{tc.err}), + )...) + + _, err := client.Register(context.Background(), + &rampv1.RegisterRequest{Exchange: domain}) + + var cerr *rampconnect.CallError + if !errors.As(err, &cerr) || cerr.Kind != tc.want { + t.Fatalf("err = %v, want kind %v", err, tc.want) + } + if origin.gotRegister != nil { + t.Fatal("a registration whose requirements read failed reached the wire") + } + }) + } +} + +// A schema this SDK refuses must not become a local veto. Refusing here would turn +// a rule about reading a third party's document into a denial of service against +// the caller's own user, so the request is sent and the Exchange decides. +func TestRegister_AnUnusableSchemaDoesNotBlockTheSend(t *testing.T) { + sig := newSigningFixture(t) + origin := &recordingAccount{billingRef: "acct-1"} + domain, _ := selfAdvertisingExchangeWith(t, sig, origin, func() map[string]any { + return map[string]any{ + "account_registration": map[string]any{ + "data_schema": json.RawMessage( + `{"$schema":"https://json-schema.org/draft/2019-09/schema","type":"object"}`), + }, + } + }) + + client := rampconnect.NewClient("http://home.invalid", + append(allowLoopback(t), rampconnect.WithSigner(sig.signer))...) + + if _, err := client.Register(context.Background(), &rampv1.RegisterRequest{ + Exchange: domain, + RegistrationData: registrationData(t, map[string]any{"anything": "goes"}), + }); err != nil { + t.Fatalf("an unusable published schema blocked the send: %v", err) + } + if origin.gotRegister == nil { + t.Fatal("the request never reached the Exchange") + } +} + +// The four bounds on registration_data are checked before anything is signed, and +// a breach is a MALFORMED request rather than a schema failure. +func TestRegister_RefusesAnOutOfBoundsPayloadBeforeSigning(t *testing.T) { + sig := newSigningFixture(t) + + tooManyMembers := map[string]any{} + for i := 0; i <= helpers.MaxRegistrationDataMembers; i++ { + tooManyMembers[string(rune('a'+i%26))+strings.Repeat("x", i)] = "v" + } + // One container per level, past the depth cap. + deep := map[string]any{"leaf": "v"} + for i := 0; i < helpers.MaxRegistrationDataDepth+2; i++ { + deep = map[string]any{"n": deep} + } + + for _, tc := range []struct { + name string + data map[string]any + }{ + {"too many top-level members", tooManyMembers}, + {"nested past the depth cap", deep}, + } { + t.Run(tc.name, func(t *testing.T) { + origin := &recordingAccount{billingRef: "acct-1"} + domain, wkHits := selfAdvertisingExchange(t, sig, origin) + client := rampconnect.NewClient("http://home.invalid", + append(allowLoopback(t), rampconnect.WithSigner(sig.signer))...) + + _, err := client.Register(context.Background(), &rampv1.RegisterRequest{ + Exchange: domain, + RegistrationData: registrationData(t, tc.data), + }) + var cerr *rampconnect.CallError + if !errors.As(err, &cerr) || cerr.Kind != rampconnect.CallMalformed { + t.Fatalf("err = %v, want a malformed CallError", err) + } + if origin.gotRegister != nil { + t.Fatal("an out-of-bounds payload reached the wire") + } + // A limit that exists to stop work runs before the work it would stop. + if n := wkHits.Load(); n != 0 { + t.Fatalf("manifest fetches = %d, want 0; the bound ran after the fetch", n) + } + }) + } +} + +// A repeat registration is answered from the stored record and returns the same +// account handle. The client neither mints nor stamps an idempotency key, because +// the message carries none. +func TestRegister_ARepeatReturnsTheSameHandle(t *testing.T) { + sig := newSigningFixture(t) + origin := &recordingAccount{billingRef: "acct-1", active: true} + domain, _ := selfAdvertisingExchange(t, sig, origin) + + client := rampconnect.NewClient("http://home.invalid", + append(allowLoopback(t), rampconnect.WithSigner(sig.signer))...) + + first, err := client.Register(context.Background(), &rampv1.RegisterRequest{Exchange: domain}) + if err != nil { + t.Fatalf("first register: %v", err) + } + second, err := client.Register(context.Background(), &rampv1.RegisterRequest{Exchange: domain}) + if err != nil { + t.Fatalf("repeat register: %v", err) + } + if first.GetBillingRef() != second.GetBillingRef() { + t.Fatalf("repeat returned %q, want %q", second.GetBillingRef(), first.GetBillingRef()) + } + if origin.registers != 2 { + t.Fatalf("Exchange saw %d registrations, want 2", origin.registers) + } +} + +// A refused registration travels as a non-OK call carrying a typed reason, and the +// caller reads it through the same accessor every other verb uses. +func TestRegister_TypedRefusalIsReadableThroughErrorDetailFrom(t *testing.T) { + sig := newSigningFixture(t) + detail := helpers.RegistrationFailureDetail( + "ramp.v1.ExchangeService", "terms have moved", + rampv1.RegistrationFailureReason_REGISTRATION_FAILURE_REASON_TERMS_DIGEST_STALE) + refusal := rampserver.AttachDetail( + connectrpc.NewError(connectrpc.CodeFailedPrecondition, errors.New("stale terms")), detail) + + origin := &recordingAccount{refuse: refusal} + domain, _ := selfAdvertisingExchange(t, sig, origin) + + client := rampconnect.NewClient("http://home.invalid", + append(allowLoopback(t), rampconnect.WithSigner(sig.signer))...) + + _, err := client.Register(context.Background(), &rampv1.RegisterRequest{Exchange: domain}) + got, ok := rampconnect.ErrorDetailFrom(err) + if !ok { + t.Fatalf("no typed detail on %v", err) + } + if got.GetRegistrationFailure().GetReason() != + rampv1.RegistrationFailureReason_REGISTRATION_FAILURE_REASON_TERMS_DIGEST_STALE { + t.Fatalf("reason = %v", got.GetRegistrationFailure().GetReason()) + } +} + +// --------------------------------------------------------------------------- +// GetAccountStatus +// --------------------------------------------------------------------------- + +// An empty account handle is a NORMAL answer: this agent holds no account there +// yet. It is not a failure and must not be reported as one. +func TestGetAccountStatus_NoAccountIsANormalAnswer(t *testing.T) { + sig := newSigningFixture(t) + origin := &recordingAccount{} + domain, _ := selfAdvertisingExchange(t, sig, origin) + + client := rampconnect.NewClient("http://home.invalid", + append(allowLoopback(t), rampconnect.WithSigner(sig.signer))...) + + resp, err := client.GetAccountStatus(context.Background(), + &rampv1.GetAccountStatusRequest{Exchange: domain}) + if err != nil { + t.Fatalf("get account status: %v", err) + } + if resp.GetBillingRef() != "" || resp.GetActive() { + t.Fatalf("response = %v, want an empty, inactive account", resp) + } + if got := origin.gotStatus.GetVer(); got != helpers.ProtocolVersion { + t.Fatalf("ver = %q, want %q", got, helpers.ProtocolVersion) + } + if got := origin.gotStatus.GetExchange(); got != domain { + t.Fatalf("exchange = %q, want %q", got, domain) + } +} + +// --------------------------------------------------------------------------- +// Both verbs +// --------------------------------------------------------------------------- + +// The recipient is the caller's to set, and a request naming none — or naming +// something that is not a bare domain — is refused before anything is signed or +// sent. The shape rule is the contract's, which the routing tier's weaker +// dialability question is not a substitute for. +func TestAccountVerbs_RefuseAnUnaddressedRequest(t *testing.T) { + sig := newSigningFixture(t) + client := rampconnect.NewClient("http://home.invalid", rampconnect.WithSigner(sig.signer)) + + for _, exchange := range []string{"", "https://exchange.test", "exchange.test/path", "ex_change.test"} { + name := exchange + if name == "" { + name = "no recipient at all" + } + t.Run(name, func(t *testing.T) { + _, err := client.Register(context.Background(), + &rampv1.RegisterRequest{Exchange: exchange}) + assertNotSent(t, err) + _, err = client.GetAccountStatus(context.Background(), + &rampv1.GetAccountStatusRequest{Exchange: exchange}) + assertNotSent(t, err) + }) + } +} + +func assertNotSent(t *testing.T, err error) { + t.Helper() + var cerr *rampconnect.CallError + if !errors.As(err, &cerr) || cerr.Kind != rampconnect.CallNotSent { + t.Fatalf("err = %v, want CallNotSent", err) + } + if cerr.Status != 0 { + t.Fatalf("a refusal that never reached the wire carries status %d", cerr.Status) + } +} + +// A nil request is a request that could not be assembled, not one the peer +// refused. +func TestAccountVerbs_RefuseANilRequest(t *testing.T) { + client := rampconnect.NewClient("http://home.invalid") + var cerr *rampconnect.CallError + + _, err := client.Register(context.Background(), nil) + if !errors.As(err, &cerr) || cerr.Kind != rampconnect.CallMalformed { + t.Fatalf("register: err = %v, want CallMalformed", err) + } + _, err = client.GetAccountStatus(context.Background(), nil) + if !errors.As(err, &cerr) || cerr.Kind != rampconnect.CallMalformed { + t.Fatalf("get account status: err = %v, want CallMalformed", err) + } +} + +// The caller's own protocol version wins: `ver` is filled only when empty. +func TestAccountVerbs_CallerVerWins(t *testing.T) { + sig := newSigningFixture(t) + origin := &recordingAccount{} + domain, _ := selfAdvertisingExchange(t, sig, origin) + client := rampconnect.NewClient("http://home.invalid", + append(allowLoopback(t), rampconnect.WithSigner(sig.signer))...) + + if _, err := client.Register(context.Background(), + &rampv1.RegisterRequest{Ver: "9.9", Exchange: domain}); err != nil { + t.Fatalf("register: %v", err) + } + if got := origin.gotRegister.GetVer(); got != "9.9" { + t.Fatalf("register ver = %q, want the caller's 9.9", got) + } + if _, err := client.GetAccountStatus(context.Background(), + &rampv1.GetAccountStatusRequest{Ver: "9.9", Exchange: domain}); err != nil { + t.Fatalf("get account status: %v", err) + } + if got := origin.gotStatus.GetVer(); got != "9.9" { + t.Fatalf("status ver = %q, want the caller's 9.9", got) + } +} + +// A signed call is never redirected: following one would re-sign the caller's +// request for a target the peer chose. +func TestAccountVerbs_RefuseARedirect(t *testing.T) { + sig := newSigningFixture(t) + domain, _ := loopbackManifestServerWith(t, http.HandlerFunc( + func(w http.ResponseWriter, _ *http.Request) { + http.Redirect(w, &http.Request{}, "http://elsewhere.invalid/", http.StatusFound) + }), nil) + + client := rampconnect.NewClient("http://home.invalid", + append(allowLoopback(t), rampconnect.WithSigner(sig.signer))...) + + _, err := client.GetAccountStatus(context.Background(), + &rampv1.GetAccountStatusRequest{Exchange: domain}) + var cerr *rampconnect.CallError + if !errors.As(err, &cerr) || cerr.Kind != rampconnect.CallUnreachable { + t.Fatalf("err = %v, want CallUnreachable", err) + } +} + +// A status request carries no varying field, so two calls inside one wall-clock +// second sign IDENTICAL bytes — signature timestamps have one-second resolution — +// and a peer screening replays on (key id, signature) refuses the second. +// +// This is a behaviour test rather than a corpus row on purpose: the hazard cannot +// be expressed as bytes, because demonstrating it needs two sequential calls +// against a server holding a replay store. +// +// The SDK does not pick the window for the caller. A window is one instance per +// CLIENT rather than per call — it carries the running maximum that makes each +// signature unique — so the choice belongs to whoever builds the client, and both +// halves of that choice are pinned here. +func TestGetAccountStatus_IdenticalRequestsCollideUnlessTheWindowMoves(t *testing.T) { + // One reading, shared by both calls, so they land in the SAME second — which is + // the condition under test. It has to be a real instant rather than a fixed + // one, because the server verifies freshness against its own clock. + now := time.Now() + frozen := func() time.Time { return now } + + serve := func(t *testing.T, sig signingFixture, svc rampv1connect.ExchangeServiceHandler) string { + t.Helper() + path, h := rampserver.NewExchangeServiceHandler(svc, + rampserver.WithKeyResolver(sig.resolver), + rampserver.WithReplayStore(newMemReplayStore())) + rpc := http.NewServeMux() + rpc.Handle(path, h) + domain, _ := loopbackManifestServerWith(t, rpc, nil) + return domain + } + + call := func(t *testing.T, window core.Window) error { + t.Helper() + sig := newSigningFixture(t) + domain := serve(t, sig, &recordingAccount{}) + client := rampconnect.NewClient("http://home.invalid", append(allowLoopback(t), + rampconnect.WithSigner(sig.signer), + rampconnect.WithSignWindow(window), + )...) + req := &rampv1.GetAccountStatusRequest{Exchange: domain} + if _, err := client.GetAccountStatus(context.Background(), req); err != nil { + t.Fatalf("first call: %v", err) + } + _, err := client.GetAccountStatus(context.Background(), req) + return err + } + + t.Run("a plain clock window collides", func(t *testing.T) { + err := call(t, core.ClockWindow(frozen, 5*time.Minute)) + if err == nil { + t.Fatal("the second identical call was accepted; the hazard this documents is gone " + + "and the verb doc plus the monotonic case below need revisiting") + } + var cerr *rampconnect.CallError + if !errors.As(err, &cerr) || cerr.Kind != rampconnect.CallRefused { + t.Fatalf("err = %v, want a refusal from the peer", err) + } + }) + + t.Run("a monotonic window does not", func(t *testing.T) { + if err := call(t, core.MonotonicWindow(frozen, 5*time.Minute)); err != nil { + t.Fatalf("the second call was refused under a monotonic window: %v", err) + } + }) +} diff --git a/sdk/go/connect/callerror.go b/sdk/go/connect/callerror.go index 85f01e04..809d3ec0 100644 --- a/sdk/go/connect/callerror.go +++ b/sdk/go/connect/callerror.go @@ -65,15 +65,54 @@ func (k CallErrorKind) String() string { return failure.Name(callErrorKindNames, // Detail carries the typed protocol reason when there is one. On an RPC path it // is the ErrorDetail the peer emitted; on the content path it is SYNTHESIZED // locally from the edge's refusal token, because the edge answers a small JSON -// object rather than a protobuf. ErrorDetailFrom reads both, so a caller branches -// on one vocabulary either way. +// object rather than a protobuf; and on a registration refused by the client's own +// pre-check it is synthesized from the schema failures that pre-check found, which +// are the failures the Exchange would have named had the request been sent. +// ErrorDetailFrom reads all three, so a caller branches on one vocabulary whichever +// side declined. type CallError struct { Kind CallErrorKind Op string Status int // HTTP status when the peer answered; 0 otherwise Reason string // the peer's own refusal token when it sent one Detail *rampv1.ErrorDetail - Err error + // PeerMessage is the developer message the peer put on its TYPED reason, when + // it sent one. Empty otherwise. + // + // It is a field rather than something to recover from Error()'s rendering + // because a reason rendered into prose cannot be read back out without + // parsing it, and a layer that has to do that is a layer that will get it + // wrong. It sits BESIDE Reason rather than in it: Reason is the peer's + // machine token, and putting prose there was a mistake this SDK has already + // made once and reverted. + // + // It is deliberately NOT filled from the transport envelope when there is no + // typed detail. An answer that did not come from a RAMP service — a draining + // load balancer, a proxy's own page — carries no message of its own, and the + // text a transport synthesizes for it is that transport's, not the peer's: + // connect-go writes "502 Bad Gateway" where a fetch-based client writes + // nothing, so carrying it would make this field's value a property of the + // language rather than of the answer. That text is still reachable through + // the cause, where it reads as what it is. + // + // It is equally NOT filled from a detail this SDK BUILT ITSELF — the content + // leg's refusal sentence, or the registration pre-check's. Those details carry + // a typed reason, so the envelope rule above would not stop them; what stops + // them is that the sentence around the reason is ours. A field that exists so a + // layer can attribute prose to a remote party cannot sometimes hold our words. + // In Go only the decode site fills this, so the rule holds structurally; the + // two ports state it where their constructors could otherwise derive it. + // + // NON-AUTHORITATIVE, and the contract says so of the field it comes from. + // Branch on Kind or on the typed reason, never on this text. It is also + // UNBOUNDED — the contract calls it an easy existence oracle and places the + // no-secrets duty on the server, so a consumer that renders it to a log line + // or to an agent bounds it there, where the audience is known. This SDK does + // not bound it, for the same reason it does not bound Detail.Message: + // truncating a peer's only account of why a call failed is a decision that + // belongs to whoever displays it. + PeerMessage string + Err error } func (e *CallError) Error() string { @@ -150,6 +189,7 @@ func sendError(op string, err error) error { out.Reason = cerr.Code().String() if detail, ok := errorDetailFromConnect(cerr); ok { out.Detail = detail + out.PeerMessage = detail.GetMessage() } return out } diff --git a/sdk/go/connect/client.go b/sdk/go/connect/client.go index 47bd1842..16100824 100644 --- a/sdk/go/connect/client.go +++ b/sdk/go/connect/client.go @@ -36,6 +36,10 @@ type Client struct { // endpoints resolves an offer's exchange domain to that Exchange's own // advertised origin. Never configuration. endpoints EndpointResolver + // requirements reads what an Exchange asks of a registration. It holds no + // document cache, which the terms-digest rule depends on — see + // WithRegistrationRequirements. + requirements RegistrationRequirementsReader // fetcher is the content leg. It dials, so it lives one tier down. fetcher *resolvers.ContentFetcher } @@ -102,7 +106,8 @@ func NewClient(baseURL string, opts ...ClientOption) *Client { // goroutine and a socket open indefinitely. exchanges: newExchangePool( offerDerivedClient(cfg, resolvers.NewGuardedTransport(cfg.guardedBase)), connectOpts...), - endpoints: cfg.resolveEndpointResolver(), + endpoints: cfg.resolveEndpointResolver(), + requirements: cfg.resolveRequirementsReader(), fetcher: resolvers.NewContentFetcher(resolvers.ContentFetchOptions{ BaseTransport: cfg.guardedBase, Timeout: cfg.fetchTimeout, diff --git a/sdk/go/connect/doc.go b/sdk/go/connect/doc.go index 8fef2ed4..e37c5399 100644 --- a/sdk/go/connect/doc.go +++ b/sdk/go/connect/doc.go @@ -1,10 +1,17 @@ // Package connect is the opt-in Connect-Go CLIENT binding over the transport-neutral // sdk/go/core L2 substance. It carries the agent verb set — Discover, Resolve, -// Execute, ReportUsage, Dispute and the low-tier content Fetch — and the publisher -// verb set — PushResources, RemoveResources and RefreshCatalog — across three -// constructors: NewClient for an Exchange, NewBrokerClient for a Broker, and -// NewCatalogClient for an Exchange's CatalogService, which are different parties -// or different addresses and so cannot share one base URL. +// Execute, ReportUsage, Dispute and the low-tier content Fetch — the account-setup +// verb set — Register and GetAccountStatus — and the publisher verb set — +// PushResources, RemoveResources and RefreshCatalog — across three constructors: +// NewClient for an Exchange, NewBrokerClient for a Broker, and NewCatalogClient +// for an Exchange's CatalogService, which are different parties or different +// addresses and so cannot share one base URL. +// +// The account verbs sit on NewClient rather than on a fourth constructor because +// they are the same party holding the same key: an account is per-Exchange and +// which Exchange is the agent's choice per call, so they route off the request's +// own exchange field exactly as a usage report does, rather than against a +// configured origin. // // Around those sit the configurable client itself (sign face as a signing // RoundTripper, request-id/validate interceptors, the fail-closed offer Verifier), @@ -14,8 +21,9 @@ // protovalidate interceptor (NewValidateInterceptor, the single definition the // server binding also composes), and the READ direction of the ADR-019 // ErrorDetail↔Connect bridge (ErrorDetailFrom — a client reads the typed error -// detail an upstream emitted, whether a peer sent it or the content leg -// synthesized it). +// detail behind a failure, whether a peer sent it, the content leg synthesized it +// from an edge's refusal token, or a registration pre-check synthesized it from the +// schema failures it found before sending). // // It depends one-directionally on core (Verifier, DiscoveryResult, VerifiedOffer // guard, signing transport, ReplayStore) and on resolvers for the one thing this diff --git a/sdk/go/connect/errordetail.go b/sdk/go/connect/errordetail.go index 2b19741e..8ecfceca 100644 --- a/sdk/go/connect/errordetail.go +++ b/sdk/go/connect/errordetail.go @@ -17,11 +17,14 @@ import ( // (AsConnectError) lives in the server binding sdk/go/connectserver. func ErrorDetailFrom(err error) (*rampv1.ErrorDetail, bool) { // The SDK's own typed failure is checked first, so ONE accessor serves every - // verb. On an RPC path the detail below was emitted by the peer; on the - // content path it was synthesized locally from the edge's refusal token, - // because a delivery edge answers a small JSON object rather than a protobuf. - // What a synthesized detail names as its domain, and why, is recorded once on - // edgeErrorDomain rather than restated here. + // verb. The detail below reaches it three ways. On an RPC path the peer emitted + // it. On the content path it was synthesized locally from the edge's refusal + // token, because a delivery edge answers a small JSON object rather than a + // protobuf. On a registration the client PRE-CHECKED and refused, it was + // synthesized from the same schema failures the Exchange would have named, so a + // caller reads one shape whichever side declined. What a synthesized detail + // names as its domain, and why, is recorded once on edgeErrorDomain rather than + // restated here. if callErr, ok := asCallError(err); ok && callErr.Detail != nil { return callErr.Detail, true } diff --git a/sdk/go/connect/gen_client_request_vectors_test.go b/sdk/go/connect/gen_client_request_vectors_test.go index 5601257b..87ee3c3a 100644 --- a/sdk/go/connect/gen_client_request_vectors_test.go +++ b/sdk/go/connect/gen_client_request_vectors_test.go @@ -45,6 +45,7 @@ import ( "github.com/RAMP-Protocol/protocol/sdk/go/core" "github.com/RAMP-Protocol/protocol/sdk/go/helpers" "github.com/RAMP-Protocol/protocol/sdk/go/internal/vectorio" + "github.com/RAMP-Protocol/protocol/sdk/go/resolvers" ) const clientRequestVectorsPath = "testdata/client-request-vectors.json" @@ -136,6 +137,12 @@ func buildClientRequestVectors(t *testing.T) []clientRequestVector { rampconnect.WithClientOptions( connectrpc.WithCodec(rampserver.EmitUnpopulatedJSONCodec()), ), + // Register reads an Exchange's published requirements before it signs. What + // this corpus records is the path and the envelope, so the read is stubbed + // out to an Exchange that publishes neither a terms digest nor a schema — + // the ordinary pass-through case, and the one that leaves the recorded body + // to the caller's own fields. Inert for every other verb. + rampconnect.WithRegistrationRequirements(publishesNothing{}), } var out []clientRequestVector @@ -193,6 +200,20 @@ func buildClientRequestVectors(t *testing.T) []clientRequestVector { }, rampconnect.WithIdempotencyKey(pinnedKey)) return err }) + capture("register", "register", func(c *rampconnect.Client, exchange string) error { + _, err := c.Register(context.Background(), &rampv1.RegisterRequest{Exchange: exchange}) + return err + }) + capture("register_caller_ver_wins", "register", func(c *rampconnect.Client, exchange string) error { + _, err := c.Register(context.Background(), + &rampv1.RegisterRequest{Exchange: exchange, Ver: "9.9"}) + return err + }) + capture("get_account_status", "getAccountStatus", func(c *rampconnect.Client, exchange string) error { + _, err := c.GetAccountStatus(context.Background(), + &rampv1.GetAccountStatusRequest{Exchange: exchange}) + return err + }) out = append(out, executeVector(t, baseOpts)) out = append(out, brokerResolveVector(t, baseOpts)) out = append(out, catalogVectors(t, baseOpts)...) @@ -352,3 +373,14 @@ func brokerResolveVector(t *testing.T, opts []rampconnect.ClientOption) clientRe RequesterID: requesterIDOf(seen.body), } } + +// publishesNothing is an Exchange that publishes neither a terms digest nor a +// registration schema — the contract's ordinary pass-through case. It stands in +// for the network read so this corpus records the envelope alone. +type publishesNothing struct{} + +func (publishesNothing) ResolveRegistrationRequirements( + context.Context, string, +) (resolvers.RegistrationRequirements, error) { + return resolvers.RegistrationRequirements{Verdict: helpers.SchemaNotPublished}, nil +} diff --git a/sdk/go/connect/gen_synthesized_detail_vectors_test.go b/sdk/go/connect/gen_synthesized_detail_vectors_test.go new file mode 100644 index 00000000..dd8d6976 --- /dev/null +++ b/sdk/go/connect/gen_synthesized_detail_vectors_test.go @@ -0,0 +1,253 @@ +package connect_test + +// Synthesized-detail golden-vector emitter: the typed details this SDK BUILDS rather +// than receives. +// +// Most ErrorDetails a client hands back were written by a peer, so what the three SDKs +// have to agree on is only how they DECODE one — which error-detail-vectors.json already +// pins. Two are different. A delivery edge answers a small JSON object rather than a +// protobuf, so the content leg promotes its refusal token into a typed reason and writes +// the envelope around it. A registration the client refuses against the Exchange's +// published schema never reaches a peer at all, so every field of that detail is ours. +// +// In both, the domain, the sentence and the reason are AUTHORED, in three languages, +// from three copies of the same decision. Nothing compared them. That is the shape a +// shared corpus exists for, and the reason the edge rows are here beside the newer ones +// even though they predate them: the domain had been a bare literal in three files, and +// the token table beneath it had silently come apart. Two of the ports had DERIVED each +// enum name from the token by uppercasing it, which is right for two of the eleven +// tokens the protocol records and wrong for the rest — so nine real edge refusals +// carried a typed reason in Go and none in either port, and every suite was green. +// Every recorded token is a row now, including the two that must stay UNTYPED. +// +// What is NOT pinned is the field errors' prose. Their PATHS are recorded, because those +// are a projection of the payload the three agree on; the constraint text beside each +// comes from a different validator library in each language and the contract calls it +// non-authoritative for exactly that reason. +// +// Vectors are captured by driving the REAL client until it refuses, so this records what +// the oracle does rather than a description of it. +// +// Verification no-op by default; (re)writes under RAMP_UPDATE_VECTORS=1. TEST +// INFRASTRUCTURE. + +import ( + "context" + "errors" + "fmt" + "net/http" + "net/http/httptest" + "os" + "testing" + + rampv1 "github.com/RAMP-Protocol/protocol/gen/go/ramp/v1" + rampconnect "github.com/RAMP-Protocol/protocol/sdk/go/connect" + "github.com/RAMP-Protocol/protocol/sdk/go/helpers" + "github.com/RAMP-Protocol/protocol/sdk/go/internal/vectorio" +) + +const synthesizedDetailVectorsPath = "testdata/synthesized-detail-vectors.json" + +// precheckVector is one registration the client refused before sending anything, and the +// typed detail that refusal carries. +type precheckVector struct { + Name string `json:"name"` + //: The data_schema the Exchange published, verbatim, as the reader would serve it. + Schema string `json:"schema"` + //: The registration_data the caller offered against that schema. + Data map[string]any `json:"data"` + //: The failing surface. A refusal computed here names the CLIENT's own tier: the + //: Exchange never saw the request, so naming it would attribute a local verdict to a + //: party that reached none. + Domain string `json:"domain"` + //: The detail's developer message. Authored by the SDK, so the three must write it + //: identically or a consumer reads two problems where there is one. + Message string `json:"message"` + //: Which typed reason block is populated, as its proto-JSON key. + ReasonField string `json:"reason_field"` + //: The reason enum's name. + ReasonEnum string `json:"reason_enum"` + //: The offending members, as RFC 6901 pointers relative to registration_data — the + //: empty pointer addressing the whole object. PATHS ONLY: the constraint prose beside + //: each comes from each language's validator library and is documented + //: non-authoritative, so pinning it here would pin an accident. + FieldErrorPaths []string `json:"field_error_paths"` +} + +// edgeRefusalVector is one delivery-edge refusal, promoted from the edge's own token into +// a typed reason the SDK wrote the envelope around. +type edgeRefusalVector struct { + Name string `json:"name"` + //: The token the edge answered in its own vocabulary. + ReasonToken string `json:"reason_token"` + //: The failing surface: the delivery edge, not the fetched resource. A per-URL value + //: has unbounded cardinality and groups nothing. EMPTY when the protocol records no + //: typed reason for this token — there is then no detail at all, and the caller falls + //: back to the failure class and the raw token, which is the honest answer. + Domain string `json:"domain"` + //: The detail's developer message. The token is the edge's; this sentence is ours. + Message string `json:"message"` + //: Which typed reason block is populated, as its proto-JSON key. + ReasonField string `json:"reason_field"` + //: The reason enum's name. + ReasonEnum string `json:"reason_enum"` +} + +func TestGenerateSynthesizedDetailVectors(t *testing.T) { + doc := map[string]any{ + "note": "Typed error details this SDK builds itself rather than receiving, " + + "captured from the real Go client. Each row records what the three " + + "languages must author identically: the failing surface, the sentence and " + + "the typed reason. Field errors are recorded as PATHS and never as text — " + + "the text comes from a different validator library in each language and the " + + "contract calls it non-authoritative.", + "registration_precheck": buildPrecheckVectors(t), + "edge_refusal": buildEdgeRefusalVectors(t), + } + if os.Getenv("RAMP_UPDATE_VECTORS") == "1" { + if err := vectorio.Write(synthesizedDetailVectorsPath, doc); err != nil { + t.Fatalf("write %s: %v", synthesizedDetailVectorsPath, err) + } + return + } + stale, err := vectorio.Stale(synthesizedDetailVectorsPath, doc) + if err != nil { + t.Fatalf("read %s: %v", synthesizedDetailVectorsPath, err) + } + if stale { + t.Fatalf("%s is stale; re-run with RAMP_UPDATE_VECTORS=1 to regenerate", + synthesizedDetailVectorsPath) + } +} + +func buildPrecheckVectors(t *testing.T) []precheckVector { + t.Helper() + cases := []struct { + name string + schema string + data map[string]any + }{ + // Both member shapes in one refusal: a required member that is ABSENT, which is a + // whole-object failure reported against the empty pointer, and a member that is + // PRESENT and breaks its pattern, reported against its own. The empty pointer is + // the one a consumer most plausibly renders wrong, so no row may omit it. + {"both_member_shapes", bothShapesSchema, map[string]any{"vat_id": "de1"}}, + // The single whole-object failure on its own, because it is the shape a first + // registration against a real Exchange actually hits. + {"whole_object_only", legalEntitySchema, map[string]any{"trading_name": "Acme"}}, + } + + out := make([]precheckVector, 0, len(cases)) + for _, c := range cases { + cerr, _ := refusedByTheSchema(t, c.schema, c.data) + detail, ok := rampconnect.ErrorDetailFrom(cerr) + if !ok { + t.Fatalf("%s: the pre-check refusal carries no typed detail", c.name) + } + field, enum := reasonProjectionOf(t, detail) + paths := make([]string, 0, len(detail.GetRegistrationFailure().GetFieldErrors())) + for _, f := range detail.GetRegistrationFailure().GetFieldErrors() { + paths = append(paths, f.GetPath()) + } + out = append(out, precheckVector{ + Name: c.name, + Schema: c.schema, + Data: c.data, + Domain: detail.GetDomain(), + Message: detail.GetMessage(), + ReasonField: field, + ReasonEnum: enum, + FieldErrorPaths: paths, + }) + } + return out +} + +func buildEdgeRefusalVectors(t *testing.T) []edgeRefusalVector { + t.Helper() + // EVERY token the protocol records, not a sample. This is a per-token lookup against + // a table, so a sample proves only that the table exists — and one language having + // computed the enum name from the token instead of reading the record, and thereby + // typing two of these and dropping the rest, is exactly what a sample would have + // missed. The order is the proto's own. + tokens := []string{ + // Signed-URL checks. + "expired", "missing_exp", "signature_mismatch", + // Proof-of-possession checks. + "missing_agent_key", "keyid_mismatch", "thumbprint_mismatch", + "pop_missing_created", "pop_missing_exp", "pop_expired", "pop_sig_invalid", + // Recorded by the proto against TWO values, so the body cannot say which check + // ran and no language may guess. It is here as a row rather than left out, + // because "deliberately untyped" is a decision a corpus should hold, and a + // language that types it has over-promoted a refusal the wire never attributed. + "missing_sig", + // Not a token any edge emits — it is the enum's own suffix, and it lands here + // only in a language that DERIVES the name from the token instead of reading the + // protocol's record. Recorded as untyped so that mistake cannot come back green. + "url_expired", + } + + out := make([]edgeRefusalVector, 0, len(tokens)) + for _, token := range tokens { + cerr := refusedByTheEdge(t, token) + vector := edgeRefusalVector{Name: "edge_" + token, ReasonToken: token} + if detail, ok := rampconnect.ErrorDetailFrom(cerr); ok { + vector.Domain = detail.GetDomain() + vector.Message = detail.GetMessage() + vector.ReasonField, vector.ReasonEnum = reasonProjectionOf(t, detail) + } + out = append(out, vector) + } + return out +} + +// refusedByTheEdge drives one bound fetch against an edge that answers a refusal token, +// and returns the client's typed failure. +func refusedByTheEdge(t *testing.T, token string) *rampconnect.CallError { + t.Helper() + sig := newSigningFixture(t) + edge := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, _ *http.Request) { + w.WriteHeader(http.StatusForbidden) + _, _ = fmt.Fprintf(w, `{"reason":%q}`, token) + })) + defer edge.Close() + + client := rampconnect.NewClient("http://home.invalid", + append(allowLoopback(t), + rampconnect.WithSigner(sig.signer), + rampconnect.WithAgentKey(sig.pub), + )...) + _, err := client.Fetch(context.Background(), edge.URL+"/doc?agent_id=tp") + + var cerr *rampconnect.CallError + if !errors.As(err, &cerr) { + t.Fatalf("edge refusal is not a typed failure: %v", err) + } + return cerr +} + +// reasonProjectionOf reads which typed reason block a detail carries and what it says. +// +// The field name comes off the DESCRIPTOR rather than a switch here, so a reason family +// added to the contract cannot be projected under a name this file invented for it. +func reasonProjectionOf(t *testing.T, detail *rampv1.ErrorDetail) (field, enum string) { + t.Helper() + reason := helpers.Reason(detail) + if reason == nil { + t.Fatal("a synthesized detail carries no typed reason; the vector would pin nothing") + } + named, ok := reason.(fmt.Stringer) + if !ok { + t.Fatalf("typed reason %T does not name itself", reason) + } + m := detail.ProtoReflect() + od := m.Descriptor().Oneofs().ByName("reason") + if od == nil { + t.Fatal("ErrorDetail has no `reason` oneof; this projection has lost its anchor") + } + fd := m.WhichOneof(od) + if fd == nil { + t.Fatal("the `reason` oneof is unset on a detail that named a reason") + } + return string(fd.Name()), named.String() +} diff --git a/sdk/go/connect/options.go b/sdk/go/connect/options.go index c31846df..6f25eb38 100644 --- a/sdk/go/connect/options.go +++ b/sdk/go/connect/options.go @@ -35,6 +35,7 @@ type clientConfig struct { signWindow core.Window signatureAgent string endpoints EndpointResolver + requirements RegistrationRequirementsReader guardedBase *http.Transport connectOpts []connectrpc.ClientOption fetchTimeout time.Duration @@ -244,6 +245,22 @@ func WithEndpointResolver(r EndpointResolver) ClientOption { return func(c *clientConfig) { c.endpoints = r } } +// WithRegistrationRequirements injects the reader that reports what one Exchange +// asks of a registration — the terms revision submitting one accepts, and the +// schema its registration_data must match. It defaults to the SSRF-guarded +// well-known reader. +// +// The reader it takes holds no document cache, and that is the point rather than +// an implementation detail: the contract requires a registering client to read the +// terms digest from a FRESHLY fetched manifest, so an implementation that serves +// this from a cache breaks the rule the field exists to record. There is +// deliberately no option to supply a digest or a schema directly — a caller that +// wants to manage its own requirements sets RegisterRequest.terms_digest, which +// suppresses the read and says so on the message the signature covers. +func WithRegistrationRequirements(r RegistrationRequirementsReader) ClientOption { + return func(c *clientConfig) { c.requirements = r } +} + // WithGuardedBaseTransport carries the caller's own transport settings — a tuned // connection pool, client certificates via TLSClientConfig — UNDERNEATH the SSRF // guard on both legs that dial an address another party named: the content fetch, @@ -322,3 +339,16 @@ func (c clientConfig) resolveEndpointResolver() EndpointResolver { } return resolvers.NewWellKnownEndpointResolver(resolvers.WellKnownOptions{}) } + +// resolveRequirementsReader returns the injected registration-requirements reader +// or the guarded default, mirroring the endpoint resolver above. +// +// Defaulted rather than required because a registration that leaves terms_digest +// unset needs one to be conformant at all, and a client that silently sent no +// digest would earn a refusal the caller could not diagnose from this side. +func (c clientConfig) resolveRequirementsReader() RegistrationRequirementsReader { + if c.requirements != nil { + return c.requirements + } + return resolvers.NewWellKnownRequirementsReader(resolvers.WellKnownOptions{}) +} diff --git a/sdk/go/connect/route.go b/sdk/go/connect/route.go index 87586230..3a1eb85b 100644 --- a/sdk/go/connect/route.go +++ b/sdk/go/connect/route.go @@ -49,6 +49,38 @@ type EndpointResolver interface { ResolveEndpoint(ctx context.Context, host string) (string, error) } +// RegistrationRequirementsReader reports what one Exchange asks of a registration. +// It is an interface for the same two reasons the endpoint seam is one: a test can +// drive a registration without standing up a manifest server, and this package has +// no way to accept a terms digest or a schema from configuration — the only way to +// skip the read is to set RegisterRequest.terms_digest, where the signature covers +// it. +// +// An implementation MUST NOT serve the answer from a cache. The contract requires a +// registering client to read the digest from a freshly fetched manifest, so a cached +// one breaks the rule the field exists to record. +// +// Its ERROR decides how a caller is told to react, as the endpoint seam's does. A +// failure that is a VERDICT — the domain is unusable, the deployment excludes it, +// the document served is not an Exchange's, or it is one this reader cannot use — +// MUST wrap helpers.ErrInvalidHost, resolvers.ErrExchangeNotPermitted, +// resolvers.ErrManifestNotExchange or resolvers.ErrManifestUnusable; those four +// surface as CallNotSent, which tells the caller not to retry. Anything else is read +// as a transport failure and reported as CallUnreachable, i.e. worth retrying. An +// implementation that returns a bare error for a refusal therefore has its final +// answer retried indefinitely. +// +// ErrManifestUnusable is the one an implementation stricter than the SDK's own +// reaches for. That reader refuses three things and treats every other +// disappointment as absence or as a transport failure; one that validates the whole +// document, or refuses a version, is holding a final answer this seam would +// otherwise report as transient. +type RegistrationRequirementsReader interface { + ResolveRegistrationRequirements( + ctx context.Context, exchange string, + ) (resolvers.RegistrationRequirements, error) +} + // vetExchangeEndpoint resolves exchangeDomain to an origin a signed call may be // sent to, or refuses — naming the check that declined, and classifying it by // CAUSE. "The Exchange said no", "we could not reach it" and "we refused to dial diff --git a/sdk/go/connect/synthesized_detail_corpus_test.go b/sdk/go/connect/synthesized_detail_corpus_test.go new file mode 100644 index 00000000..0c6fbe59 --- /dev/null +++ b/sdk/go/connect/synthesized_detail_corpus_test.go @@ -0,0 +1,112 @@ +package connect_test + +// The oracle's own replay of the synthesized-detail corpus. +// +// The emitter beside this file already fails when the committed bytes drift from what the +// client produces, so equality is not what this adds. What it adds is the OBLIGATIONS the +// corpus has to keep to be worth replaying at all: that both member shapes are exercised, +// that a recorded typed reason really is the one the SDK's own token table names, and +// that the untyped rows stay untyped. A corpus can be regenerated into agreeing with a +// regression; it cannot be regenerated into satisfying these. + +import ( + "encoding/json" + "os" + "slices" + "testing" + + "github.com/RAMP-Protocol/protocol/sdk/go/helpers" +) + +func loadSynthesizedDetailCorpus(t *testing.T) ([]precheckVector, []edgeRefusalVector) { + t.Helper() + b, err := os.ReadFile(synthesizedDetailVectorsPath) + if err != nil { + t.Fatalf("read %s: %v", synthesizedDetailVectorsPath, err) + } + var doc struct { + Precheck []precheckVector `json:"registration_precheck"` + Edge []edgeRefusalVector `json:"edge_refusal"` + } + if err := json.Unmarshal(b, &doc); err != nil { + t.Fatalf("parse %s: %v", synthesizedDetailVectorsPath, err) + } + if len(doc.Precheck) == 0 || len(doc.Edge) == 0 { + t.Fatalf("%s carries an empty case list; this replay would assert nothing", + synthesizedDetailVectorsPath) + } + return doc.Precheck, doc.Edge +} + +// The pre-check rows have to carry BOTH member shapes somewhere, because they are +// rendered differently and one of them was rendered wrong: an empty pointer addresses the +// whole object, and writing "path: error" for it produced a member with no name. A corpus +// that only ever recorded named members would have pinned the bug. +func TestSynthesizedDetailCorpus_CarriesBothMemberShapes(t *testing.T) { + precheck, _ := loadSynthesizedDetailCorpus(t) + + var sawRoot, sawMember bool + for _, v := range precheck { + if len(v.FieldErrorPaths) == 0 { + t.Errorf("%s: a refusal row names no offending member", v.Name) + } + for _, p := range v.FieldErrorPaths { + if p == "" { + sawRoot = true + } else { + sawMember = true + } + } + if v.Domain == "" || v.Message == "" || v.ReasonField == "" { + t.Errorf("%s: a refusal row records no failing surface, sentence or reason", v.Name) + } + } + if !sawRoot { + t.Error("no row carries the empty pointer, so nothing pins how a whole-object " + + "failure is reported") + } + if !sawMember { + t.Error("no row carries a member pointer, so nothing pins the ordinary case") + } +} + +// A recorded reason is the one the SDK's own token table names, and a row recorded as +// untyped is one the table deliberately does not name. Read against the table rather than +// restated, so a token that gains or loses a mapping cannot leave the corpus asserting the +// old answer. +func TestSynthesizedDetailCorpus_EdgeRowsAgreeWithTheTokenTable(t *testing.T) { + _, edge := loadSynthesizedDetailCorpus(t) + + var typed int + for _, v := range edge { + reason, mapped := helpers.RetrievalAuthFailureReasonFromToken(v.ReasonToken) + switch { + case mapped && v.ReasonEnum != reason.String(): + t.Errorf("%s: recorded %q, table names %q", v.Name, v.ReasonEnum, reason) + case mapped && v.ReasonField != "retrieval_auth_failure": + t.Errorf("%s: a mapped token recorded reason field %q", v.Name, v.ReasonField) + case mapped && v.Domain == "": + t.Errorf("%s: a mapped token recorded no failing surface", v.Name) + case !mapped && (v.ReasonEnum != "" || v.Domain != ""): + t.Errorf("%s: %q is not in the token table, so no detail may be recorded for it", + v.Name, v.ReasonToken) + } + if mapped { + typed++ + } + } + + // Both verdicts have to be present. Rows that are all typed prove nothing about + // over-promotion, and a token no edge emits must not acquire a reason from the shape + // of its own spelling. + if typed == 0 || typed == len(edge) { + t.Errorf("edge rows record only one verdict (%d typed of %d); the corpus must "+ + "carry both what is promoted and what deliberately is not", typed, len(edge)) + } + if !slices.ContainsFunc(edge, func(v edgeRefusalVector) bool { + return v.ReasonToken == "missing_sig" + }) { + t.Error("no row for the token the protocol records against TWO values; that is the " + + "case where guessing is the failure, so it is the one a corpus must hold") + } +} diff --git a/sdk/go/connect/taxonomy_test.go b/sdk/go/connect/taxonomy_test.go index 023952ea..4552639e 100644 --- a/sdk/go/connect/taxonomy_test.go +++ b/sdk/go/connect/taxonomy_test.go @@ -13,6 +13,7 @@ import ( rampv1 "github.com/RAMP-Protocol/protocol/gen/go/ramp/v1" rampconnect "github.com/RAMP-Protocol/protocol/sdk/go/connect" + rampserver "github.com/RAMP-Protocol/protocol/sdk/go/connectserver" "github.com/RAMP-Protocol/protocol/sdk/go/helpers" "github.com/RAMP-Protocol/protocol/sdk/go/resolvers" ) @@ -213,3 +214,114 @@ func TestSendError_CallerCancellationIsNotARefusal(t *testing.T) { t.Errorf("kind = %v, want CallUnreachable — the caller gave up; the peer did not refuse", cerr.Kind) } } + +// The peer's own sentence reaches a caller as a VALUE, not as something to parse +// back out of a rendered error. +// +// It is filled from a typed detail the PEER emitted, and from nothing else. Two +// other things look like candidates and are not: the transport envelope's prose, +// which a transport synthesizes and so is a property of the language rather than +// of the answer, and a detail this SDK built ITSELF on the content leg, whose +// message is our sentence with the edge's token quoted into it. The field exists +// so a consumer can attribute words to a remote party; a field that sometimes +// holds our own words cannot do that job at all. +func TestCallError_CarriesThePeersSentence(t *testing.T) { + t.Run("a typed detail's message wins", func(t *testing.T) { + sig := newSigningFixture(t) + detail := rampserver.NewErrorDetail( + "ramp.v1.ExchangeService", "the account is not active", nil) + refusal := rampserver.AttachDetail(connectrpc.NewError( + connectrpc.CodeFailedPrecondition, errors.New("envelope prose")), detail) + domain, _ := selfAdvertisingExchange(t, sig, &recordingAccount{refuse: refusal}) + + client := rampconnect.NewClient("http://home.invalid", + append(allowLoopback(t), rampconnect.WithSigner(sig.signer))...) + _, err := client.GetAccountStatus(context.Background(), + &rampv1.GetAccountStatusRequest{Exchange: domain}) + + assertPeerMessage(t, err, "the account is not active") + }) + + // An answer that did not come from a RAMP service carries no message of its + // own. The text a transport synthesizes for one is that transport's — connect-go + // writes a status line where a fetch-based client writes nothing — so carrying + // it would make the field's value a property of the language. It stays + // reachable through the cause, where it reads as what it is. + t.Run("no typed detail leaves it empty", func(t *testing.T) { + sig := newSigningFixture(t) + domain, _ := loopbackManifestServer(t, http.HandlerFunc( + func(w http.ResponseWriter, _ *http.Request) { + w.Header().Set("Content-Type", "application/json") + w.WriteHeader(http.StatusServiceUnavailable) + _, _ = w.Write([]byte(`{"code":"unavailable","message":"draining"}`)) + })) + + client := rampconnect.NewClient("http://home.invalid", + append(allowLoopback(t), rampconnect.WithSigner(sig.signer))...) + _, err := client.GetAccountStatus(context.Background(), + &rampv1.GetAccountStatusRequest{Exchange: domain}) + + assertPeerMessage(t, err, "") + // The transport's own account is not lost, only kept out of the field that + // claims to hold the peer's words. + if !strings.Contains(err.Error(), "draining") { + t.Fatalf("the transport's text is gone entirely: %v", err) + } + }) + + // The content leg builds a typed detail LOCALLY, out of the edge's refusal + // token: the token is the edge's, the sentence around it is this SDK's. So a + // detail is present and the peer's message is still empty — which is the case + // that separates "fill it from the detail" from "fill it where the peer's own + // answer was decoded", and the only one where the two differ. + t.Run("a detail this SDK synthesized leaves it empty", func(t *testing.T) { + sig := newSigningFixture(t) + edge := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, _ *http.Request) { + w.WriteHeader(http.StatusForbidden) + _, _ = w.Write([]byte(`{"reason":"pop_expired"}`)) + })) + defer edge.Close() + + client := rampconnect.NewClient("http://home.invalid", + append(allowLoopback(t), + rampconnect.WithSigner(sig.signer), + rampconnect.WithAgentKey(sig.pub), + )...) + _, err := client.Fetch(context.Background(), edge.URL+"/doc?agent_id=tp") + + // The typed reason still reaches the caller — this is not about losing it. + if _, ok := rampconnect.ErrorDetailFrom(err); !ok { + t.Fatalf("no typed detail on a refused fetch: %v", err) + } + assertPeerMessage(t, err, "") + }) + + // The registration pre-check is the same rule with no peer involved at all: the + // request never left the process, so there is no remote party whose words this + // field could hold. A detail is present because the schema failures are worth + // carrying; the sentence around them is ours. + t.Run("a detail the registration pre-check synthesized leaves it empty", func(t *testing.T) { + cerr, _ := refusedByTheSchema(t, legalEntitySchema, map[string]any{"trading_name": "Acme"}) + + if _, ok := rampconnect.ErrorDetailFrom(cerr); !ok { + t.Fatalf("no typed detail on a refused registration: %v", cerr) + } + assertPeerMessage(t, cerr, "") + }) +} + +func assertPeerMessage(t *testing.T, err error, want string) { + t.Helper() + var cerr *rampconnect.CallError + if !errors.As(err, &cerr) { + t.Fatalf("not a typed failure: %v", err) + } + if cerr.PeerMessage != want { + t.Fatalf("peer message = %q, want %q", cerr.PeerMessage, want) + } + // The token stays a token: prose never leaks into the field a caller branches + // on, which is the mistake this SDK has already made once and reverted. + if strings.Contains(cerr.Reason, " ") { + t.Fatalf("reason %q is prose, not a machine token", cerr.Reason) + } +} diff --git a/sdk/go/connect/testdata/client-request-vectors.json b/sdk/go/connect/testdata/client-request-vectors.json index a19c896b..a18e73da 100644 --- a/sdk/go/connect/testdata/client-request-vectors.json +++ b/sdk/go/connect/testdata/client-request-vectors.json @@ -46,6 +46,33 @@ "key_minted": false, "requester_id": "" }, + { + "name": "register", + "verb": "register", + "path": "/ramp.v1.ExchangeService/Register", + "ver": "1.0", + "idempotency_key": "", + "key_minted": false, + "requester_id": "" + }, + { + "name": "register_caller_ver_wins", + "verb": "register", + "path": "/ramp.v1.ExchangeService/Register", + "ver": "9.9", + "idempotency_key": "", + "key_minted": false, + "requester_id": "" + }, + { + "name": "get_account_status", + "verb": "getAccountStatus", + "path": "/ramp.v1.ExchangeService/GetAccountStatus", + "ver": "1.0", + "idempotency_key": "", + "key_minted": false, + "requester_id": "" + }, { "name": "execute_key_pinned", "verb": "execute", diff --git a/sdk/go/connect/testdata/synthesized-detail-vectors.json b/sdk/go/connect/testdata/synthesized-detail-vectors.json new file mode 100644 index 00000000..aaaca192 --- /dev/null +++ b/sdk/go/connect/testdata/synthesized-detail-vectors.json @@ -0,0 +1,132 @@ +{ + "edge_refusal": [ + { + "name": "edge_expired", + "reason_token": "expired", + "domain": "ramp.v1.Edge", + "message": "delivery refused: expired", + "reason_field": "retrieval_auth_failure", + "reason_enum": "RETRIEVAL_AUTH_FAILURE_REASON_URL_EXPIRED" + }, + { + "name": "edge_missing_exp", + "reason_token": "missing_exp", + "domain": "ramp.v1.Edge", + "message": "delivery refused: missing_exp", + "reason_field": "retrieval_auth_failure", + "reason_enum": "RETRIEVAL_AUTH_FAILURE_REASON_URL_EXPIRY_MISSING" + }, + { + "name": "edge_signature_mismatch", + "reason_token": "signature_mismatch", + "domain": "ramp.v1.Edge", + "message": "delivery refused: signature_mismatch", + "reason_field": "retrieval_auth_failure", + "reason_enum": "RETRIEVAL_AUTH_FAILURE_REASON_URL_SIGNATURE_MISMATCH" + }, + { + "name": "edge_missing_agent_key", + "reason_token": "missing_agent_key", + "domain": "ramp.v1.Edge", + "message": "delivery refused: missing_agent_key", + "reason_field": "retrieval_auth_failure", + "reason_enum": "RETRIEVAL_AUTH_FAILURE_REASON_AGENT_KEY_MISSING" + }, + { + "name": "edge_keyid_mismatch", + "reason_token": "keyid_mismatch", + "domain": "ramp.v1.Edge", + "message": "delivery refused: keyid_mismatch", + "reason_field": "retrieval_auth_failure", + "reason_enum": "RETRIEVAL_AUTH_FAILURE_REASON_KEYID_MISMATCH" + }, + { + "name": "edge_thumbprint_mismatch", + "reason_token": "thumbprint_mismatch", + "domain": "ramp.v1.Edge", + "message": "delivery refused: thumbprint_mismatch", + "reason_field": "retrieval_auth_failure", + "reason_enum": "RETRIEVAL_AUTH_FAILURE_REASON_THUMBPRINT_MISMATCH" + }, + { + "name": "edge_pop_missing_created", + "reason_token": "pop_missing_created", + "domain": "ramp.v1.Edge", + "message": "delivery refused: pop_missing_created", + "reason_field": "retrieval_auth_failure", + "reason_enum": "RETRIEVAL_AUTH_FAILURE_REASON_PROOF_CREATED_MISSING" + }, + { + "name": "edge_pop_missing_exp", + "reason_token": "pop_missing_exp", + "domain": "ramp.v1.Edge", + "message": "delivery refused: pop_missing_exp", + "reason_field": "retrieval_auth_failure", + "reason_enum": "RETRIEVAL_AUTH_FAILURE_REASON_PROOF_EXPIRY_MISSING" + }, + { + "name": "edge_pop_expired", + "reason_token": "pop_expired", + "domain": "ramp.v1.Edge", + "message": "delivery refused: pop_expired", + "reason_field": "retrieval_auth_failure", + "reason_enum": "RETRIEVAL_AUTH_FAILURE_REASON_PROOF_EXPIRED" + }, + { + "name": "edge_pop_sig_invalid", + "reason_token": "pop_sig_invalid", + "domain": "ramp.v1.Edge", + "message": "delivery refused: pop_sig_invalid", + "reason_field": "retrieval_auth_failure", + "reason_enum": "RETRIEVAL_AUTH_FAILURE_REASON_PROOF_SIGNATURE_INVALID" + }, + { + "name": "edge_missing_sig", + "reason_token": "missing_sig", + "domain": "", + "message": "", + "reason_field": "", + "reason_enum": "" + }, + { + "name": "edge_url_expired", + "reason_token": "url_expired", + "domain": "", + "message": "", + "reason_field": "", + "reason_enum": "" + } + ], + "note": "Typed error details this SDK builds itself rather than receiving, captured from the real Go client. Each row records what the three languages must author identically: the failing surface, the sentence and the typed reason. Field errors are recorded as PATHS and never as text — the text comes from a different validator library in each language and the contract calls it non-authoritative.", + "registration_precheck": [ + { + "name": "both_member_shapes", + "schema": "{\"type\":\"object\",\"required\":[\"legal_entity\"],\"properties\":{\"vat_id\":{\"type\":\"string\",\"pattern\":\"^[A-Z]{2}[0-9]+$\"}}}", + "data": { + "vat_id": "de1" + }, + "domain": "ramp.v1.Client", + "message": "registration_data does not match the published data_schema", + "reason_field": "registration_failure", + "reason_enum": "REGISTRATION_FAILURE_REASON_INVALID_REGISTRATION_DATA", + "field_error_paths": [ + "", + "/vat_id" + ] + }, + { + "name": "whole_object_only", + "schema": "{\"type\":\"object\",\"required\":[\"legal_entity\"],\"properties\":{\"legal_entity\":{\"type\":\"string\"}}}", + "data": { + "trading_name": "Acme" + }, + "domain": "ramp.v1.Client", + "message": "registration_data does not match the published data_schema", + "reason_field": "registration_failure", + "reason_enum": "REGISTRATION_FAILURE_REASON_INVALID_REGISTRATION_DATA", + "field_error_paths": [ + "" + ] + } + ] +} diff --git a/sdk/go/connect/testsupport_test.go b/sdk/go/connect/testsupport_test.go index f1114c05..01dd1e9d 100644 --- a/sdk/go/connect/testsupport_test.go +++ b/sdk/go/connect/testsupport_test.go @@ -39,6 +39,22 @@ import ( // It also returns the well-known FETCH COUNT, which is how the caching tests tell // a second resolve that hit the cache from one that went back to the network. func loopbackManifestServer(t *testing.T, rest http.Handler) (string, *atomic.Int64) { + t.Helper() + return loopbackManifestServerWith(t, rest, nil) +} + +// loopbackManifestServerWith is the same server with extra manifest members — +// terms_digest and the account_registration block, which the account verbs read +// and nothing else does. The extras are read per request rather than captured, so +// a test can revise the served document between calls and prove a read went back +// to the network. +// +// The body always names the Exchange role, because a real manifest does: the +// contract makes the field required, and the registration reader refuses a +// document that omits it. +func loopbackManifestServerWith( + t *testing.T, rest http.Handler, extra func() map[string]any, +) (string, *atomic.Int64) { t.Helper() var ( hits atomic.Int64 @@ -47,7 +63,17 @@ func loopbackManifestServer(t *testing.T, rest http.Handler) (string, *atomic.In mux := http.NewServeMux() mux.HandleFunc("/.well-known/ramp.json", func(w http.ResponseWriter, _ *http.Request) { hits.Add(1) - _ = json.NewEncoder(w).Encode(map[string]any{"ver": helpers.WellKnownManifestVersion, "endpoint": origin}) + doc := map[string]any{ + "ver": helpers.WellKnownManifestVersion, + "endpoint": origin, + "role": "ROLE_EXCHANGE", + } + if extra != nil { + for k, v := range extra() { + doc[k] = v + } + } + _ = json.NewEncoder(w).Encode(doc) }) mux.Handle("/", rest) srv := httptest.NewServer(mux) diff --git a/sdk/go/connect/verbs.go b/sdk/go/connect/verbs.go index a4142630..1989a476 100644 --- a/sdk/go/connect/verbs.go +++ b/sdk/go/connect/verbs.go @@ -24,6 +24,15 @@ const defaultProofWindow = 30 * time.Second // edgeErrorDomain is the ErrorDetail domain for a refusal by a delivery edge. It // is the failing surface, not the fetched resource: the field is a stable // grouping key for tooling, so it names the tier that refused. +// +// This is also where the naming rule for every domain this SDK writes is recorded, +// because a value that groups nothing consistently groups nothing at all. A domain +// suffixed Service names an RPC service that EXISTS in the contract — an answer +// from ramp.v1.ExchangeService came from that service. A domain that is a bare noun +// names a TIER that is not a service and never appears in the descriptor: this one, +// for a delivery edge, and clientErrorDomain for a refusal the client computed +// before sending. The suffix is what tells a reader which of the two they have, so +// a conformance guard holds every committed domain to it. const edgeErrorDomain = "ramp.v1.Edge" // ReportUsage files a usage report with the Exchange that ISSUED the offer — diff --git a/sdk/go/connect/verbs_test.go b/sdk/go/connect/verbs_test.go index ad182631..8b8ab7fa 100644 --- a/sdk/go/connect/verbs_test.go +++ b/sdk/go/connect/verbs_test.go @@ -464,13 +464,24 @@ func TestBrokerResolve_RefusesARequesterlessRequestLocally(t *testing.T) { // from. It returns the bare domain a report routes on, plus the well-known fetch // counter. func selfAdvertisingExchange(t *testing.T, sig signingFixture, svc rampv1connect.ExchangeServiceHandler) (string, *atomic.Int64) { + t.Helper() + return selfAdvertisingExchangeWith(t, sig, svc, nil) +} + +// selfAdvertisingExchangeWith is the same host with extra manifest members, for +// the account verbs, which read terms_digest and the published data_schema out of +// the very document that advertises the endpoint. +func selfAdvertisingExchangeWith( + t *testing.T, sig signingFixture, svc rampv1connect.ExchangeServiceHandler, + extra func() map[string]any, +) (string, *atomic.Int64) { t.Helper() path, h := rampserver.NewExchangeServiceHandler(svc, rampserver.WithKeyResolver(sig.resolver)) rpc := http.NewServeMux() rpc.Handle(path, h) // The manifest half is loopbackManifestServer's, so the self-advertising part // is written once. This is the "a real RPC handler" case its catch-all takes. - return loopbackManifestServer(t, rpc) + return loopbackManifestServerWith(t, rpc, extra) } // crossHost serves a manifest advertising SOMEONE ELSE's origin — the case the @@ -508,6 +519,10 @@ func allowLoopback(t *testing.T) []rampconnect.ClientOption { return []rampconnect.ClientOption{ rampconnect.WithEndpointResolver(resolvers.NewWellKnownEndpointResolver( resolvers.WellKnownOptions{Scheme: "http", HTTP: http.DefaultClient})), + // The registration reader dials the same manifest over the same scheme, so + // it takes the same loopback posture. Inert for every verb but Register. + rampconnect.WithRegistrationRequirements(resolvers.NewWellKnownRequirementsReader( + resolvers.WellKnownOptions{Scheme: "http", HTTP: http.DefaultClient})), } } diff --git a/sdk/go/core/sigwindow.go b/sdk/go/core/sigwindow.go index a8a048ed..e24a0333 100644 --- a/sdk/go/core/sigwindow.go +++ b/sdk/go/core/sigwindow.go @@ -37,6 +37,11 @@ func ClockWindow(now func() time.Time, ttl time.Duration) Window { // Safe for concurrent RoundTrips: the running maximum is held in an atomic // updated by compare-and-swap. To adapt an application clock interface with a // Now() method, pass the method value: MonotonicWindow(clk.Now, ttl). +// +// ONE INSTANCE PER CLIENT, never one per call. The running maximum is the whole +// mechanism: a window minted per request starts from zero, cannot see the +// previous signature, and provides exactly none of the uniqueness it was chosen +// for — while still looking correct at the call site. func MonotonicWindow(now func() time.Time, ttl time.Duration) Window { var lastExpires atomic.Int64 return func() (int64, int64) { diff --git a/sdk/go/helpers/gen_errordetail_vectors_test.go b/sdk/go/helpers/gen_errordetail_vectors_test.go index ece0b02d..46a693f2 100644 --- a/sdk/go/helpers/gen_errordetail_vectors_test.go +++ b/sdk/go/helpers/gen_errordetail_vectors_test.go @@ -233,7 +233,7 @@ func buildErrorDetailVectors(t *testing.T) []errorDetailVector { rampv1.CatalogRejectionReason_CATALOG_REJECTION_REASON_TENANT_MISMATCH) registrationFailure := RegistrationFailureDetail( - "ramp.v1.RegistrationService", "domain not verified", + "ramp.v1.ExchangeService", "domain not verified", rampv1.RegistrationFailureReason_REGISTRATION_FAILURE_REASON_DOMAIN_NOT_VERIFIED) // The schema-enforcement refusal: the one detail that carries per-member diff --git a/sdk/go/helpers/testdata/error-detail-vectors.json b/sdk/go/helpers/testdata/error-detail-vectors.json index e5574a40..6893d2c3 100644 --- a/sdk/go/helpers/testdata/error-detail-vectors.json +++ b/sdk/go/helpers/testdata/error-detail-vectors.json @@ -136,14 +136,14 @@ }, { "name": "registration_failure_reason", - "domain": "ramp.v1.RegistrationService", + "domain": "ramp.v1.ExchangeService", "message": "domain not verified", "metadata": null, "reason_field": "registration_failure", "reason_enum": "REGISTRATION_FAILURE_REASON_DOMAIN_NOT_VERIFIED", "field_errors": null, "wire_json": { - "domain": "ramp.v1.RegistrationService", + "domain": "ramp.v1.ExchangeService", "message": "domain not verified", "registration_failure": { "reason": "REGISTRATION_FAILURE_REASON_DOMAIN_NOT_VERIFIED" diff --git a/sdk/go/resolvers/endpointresolver.go b/sdk/go/resolvers/endpointresolver.go index a81a7731..64fa9c6d 100644 --- a/sdk/go/resolvers/endpointresolver.go +++ b/sdk/go/resolvers/endpointresolver.go @@ -76,6 +76,29 @@ type wellKnownDoc struct { jose.JSONWebKeySet Ver json.RawMessage `json:"ver"` Endpoint string `json:"endpoint"` + + // Role, terms_digest and the account_registration block are read by the + // registration-requirements face alone. They ride in this shared projection + // because one fetch decodes the whole document and a second struct over the + // same bytes would be a second place for the member names to drift. + // + // All three are RAW, for the same reason `ver` above is: a typed field here + // would make every face fail on a member only one of them reads. A manifest + // publishing terms_digest as a number would fail the whole decode, and a + // decode failure is classified as a transport failure — retryable — so an + // Exchange with one off-spec member would turn every usage report and every + // key resolution against it into an endless retry. Raw bytes move that + // question to the face that asks it, where a member of the wrong type is + // simply absent. See termsDigest and registrationSchemaBytes, which is also + // where the two ports already put it. + // + // NONE of them is ever cached: the endpoint cache stores the endpoint string + // and nothing else, so a caller cannot reach a stale digest through any face + // in this package. That is a property the terms_digest rule depends on — see + // WellKnownRequirementsReader. + Role json.RawMessage `json:"role"` + TermsDigest json.RawMessage `json:"terms_digest"` + AccountRegistration json.RawMessage `json:"account_registration"` } // manifestVer is the string the version rule reads out of the raw `ver` member. @@ -91,6 +114,17 @@ func manifestVer(raw json.RawMessage) string { return s } +// accountRegistration is the manifest block describing how to open an account. +// Only data_schema is read; field 2 is reserved in the contract for a future web +// registration mode, which this SDK does not act on. +type accountRegistration struct { + // DataSchema is kept as raw bytes because every cap the registration-schema + // rules state is defined over the bytes AS SERVED. Decoding and re-encoding + // would change the length the size cap measures and the number formatting the + // canonical form pins. + DataSchema json.RawMessage `json:"data_schema"` +} + // fetchWellKnownDoc GETs url and decodes the well-known manifest. It is the one // HTTP+decode path both resolvers share, so the request/status/decode handling // cannot drift between the key and endpoint faces. diff --git a/sdk/go/resolvers/registrationrequirements.go b/sdk/go/resolvers/registrationrequirements.go new file mode 100644 index 00000000..5b75d7b6 --- /dev/null +++ b/sdk/go/resolvers/registrationrequirements.go @@ -0,0 +1,271 @@ +package resolvers + +import ( + "context" + "encoding/json" + "errors" + "fmt" + "net/http" + + "github.com/RAMP-Protocol/protocol/sdk/go/helpers" +) + +// ErrExchangeNotPermitted signals that the deployment's Allow overlay excluded +// this domain. It is raised BEFORE anything is dialled, so it says nothing about +// whether the Exchange exists or answers — only that this deployment declined to +// ask. Distinct from a transport failure because the remedy is a configuration +// change here, not a retry. +var ErrExchangeNotPermitted = errors.New("resolvers: exchange domain not permitted by policy") + +// ErrManifestNotExchange signals that the document served at the domain's +// well-known path describes some other role. Registration requirements are an +// Exchange's to publish, so a manifest claiming to be an agent, a broker or a +// publisher is refused rather than read for members it has no business carrying. +// A manifest naming no role at all is refused the same way: the field is required +// by the contract, and treating an absent role as "probably an Exchange" would +// make the check advisory. +var ErrManifestNotExchange = errors.New("resolvers: well-known manifest does not describe an exchange") + +// ErrManifestUnusable signals that the document arrived and this reader cannot +// use it. It is a VERDICT and not a failed read: the bytes were served, and the +// next attempt gets the same ones, so a caller told to retry retries forever. +// +// This reader never returns it. Its own refusals are the three above, and the two +// ways a manifest can disappoint it are both deliberate non-errors: a member +// carrying a type the contract does not admit reads as ABSENT, because the +// projection is shared with the endpoint and key faces and one off-spec member +// must not fail a document the other two would have read; and a document that +// does not decode at all is a transport failure, because a proxy serving an error +// page under a 200 may well not be serving one on the next try. +// +// It exists because this seam is injectable and a reader stricter than this one +// has to be able to say so. A reader that validates the whole manifest against a +// schema, or refuses a version, holds a refusal that is final and had no way to +// declare it — and a refusal a caller cannot tell from an outage is one it retries +// against a third party's origin until something else stops it. +var ErrManifestUnusable = errors.New("resolvers: well-known manifest cannot be used by this reader") + +// RegistrationRequirements is what one Exchange asks of a registration: the terms +// revision that submitting one accepts, and the schema its registration_data must +// match. +// +// Both members are optional in the contract, and their absence is a normal answer +// rather than a failure — an Exchange that publishes neither accepts registration +// data uninspected and records no terms acceptance. +type RegistrationRequirements struct { + // TermsDigest is the manifest's terms_digest, nil when the Exchange publishes + // none. Copy it onto RegisterRequest.terms_digest unchanged: the request + // signature covers the echo, and that echo is the durable record of which + // terms revision the operator accepted. + // + // A pointer rather than a string because absent and empty are different + // answers on the wire, and only the pointer can carry the difference. + TermsDigest *string + + // Schema validates registration_data before anything is signed. It is nil in + // TWO cases — the Exchange publishes none, and the Exchange publishes one this + // SDK refuses — and Verdict is what tells them apart. Both are deliberately + // the same VALUE, because a nil *helpers.RegistrationSchema reports no + // failures and that is the behaviour the contract requires of a client in both + // cases: a local check that cannot run must not become a local veto. + Schema *helpers.RegistrationSchema + + // Verdict is the SDK's answer for the published schema. SchemaNotPublished is + // the ordinary absent case; SchemaAccepted means Schema is usable; anything + // else names why a published schema was refused, which is worth logging and is + // never worth refusing the registration over. + Verdict helpers.SchemaVerdict +} + +// WellKnownRequirementsReader reads registration requirements out of an +// Exchange's own /.well-known/ramp.json. +// +// # Every read is a fresh fetch, and that is the whole design +// +// The protocol requires it in as many words: a registering client MUST read the +// terms digest from a freshly fetched manifest rather than a cached copy. A +// cached ENDPOINT is fine — a wrong one fails loudly — but a cached DIGEST is +// not, because a client cannot detect staleness locally, so a warm cache would +// make it echo a value the Exchange has already stopped accepting and retry the +// same refusal until the cache expired. +// +// That is why this is a SEPARATE reader rather than a third face on +// WellKnownEndpointResolver. That resolver is built out of exactly the mechanism +// this value may not touch — a per-host LRU with a TTL, and single-flight +// coalescing on top of it — and it exposes no bypass. Adding a member to the +// document it decodes would satisfy the rule's letter at the one point it fails +// in practice: the first caller to reuse the SDK's own manifest path would be +// handed a digest minutes old with nothing in the API to warn them. A reader that +// holds no document cache leaves no cache slot to reuse, which is the same +// structural argument the report leg makes about configuration — the rule holds +// because there is nowhere for the wrong answer to come from, not because callers +// remember. +// +// Reading the schema out of the same fresh bytes then costs nothing extra and +// removes a failure mode of its own: a stale local schema cannot refuse a payload +// the Exchange would have taken, because there is no stale local schema. +// +// # What is deliberately NOT held +// +// Not even the compiled validator. Compiling an attacker-authored schema is +// bounded but not free, and an application registering repeatedly at the same +// Exchange may reasonably want to pay that once — but memoising it is a cache, +// which this tier does not do by default, and the useful key for it (the digest +// of the served bytes) is a property of a deployment's threat model rather than +// of the protocol. An application that wants it wraps this reader. +type WellKnownRequirementsReader struct { + http *http.Client + scheme string + allow func(domain string) bool +} + +// NewWellKnownRequirementsReader returns a reader over WellKnownOptions. Zero-value +// options are safe defaults: https scheme and the SSRF-GUARDED client. +// +// The domain is CALLER-NAMED — an agent registers at whichever Exchange it means +// to transact with, and that domain routinely arrives at runtime rather than from +// configuration — so this is the same threat shape as the endpoint resolver's +// fetch and takes the same guarded default, NOT http.DefaultClient. A deployment +// that must reach a private or loopback Exchange injects its own client via +// opts.HTTP or opts out through the SKIP_SSRF / ALLOW_INSECURE env flags. +// +// opts.TTL and opts.Now are accepted and ignored: this reader caches nothing, so +// it has no freshness to compute. They are not errors, so one WellKnownOptions +// value can build every face in this package. +func NewWellKnownRequirementsReader(opts WellKnownOptions) *WellKnownRequirementsReader { + client := opts.HTTP + if client == nil { + client = NewGuardedClientFromEnv() + } + scheme := opts.Scheme + if scheme == "" { + scheme = "https" + } + return &WellKnownRequirementsReader{http: client, scheme: scheme, allow: opts.Allow} +} + +// ResolveRegistrationRequirements fetches exchange's manifest and reports what it +// asks of a registration. The answer is never served from a cache; see the type +// doc for why that is the point rather than an omission. +// +// Two refusals come before anything is dialled, and both are here rather than at +// the call site for the reason the endpoint resolver gives for its own: they are +// properties of building this URL and of this deployment's policy, not of any one +// caller's plans, and a second caller — or one that forgot — would otherwise fetch +// whatever domain an authenticated agent named. +// +// The SHAPE predicate is IsBareDomain, the contract's rule, and not the routing +// tier's IsBareHost. The two are kept apart deliberately and this leg is where the +// difference bites: nothing upstream of here has run the contract's rule, and the +// URL below is built by concatenation, so a value carrying a path or userinfo +// would choose WHAT is fetched rather than merely where from. A trailing root dot, +// a leading or trailing hyphen, an underscore and a bracketed IPv6 literal are all +// usable hosts that the wire rule refuses. +// +// A refused schema is never an error. The verdict is returned alongside a nil +// Schema, because the contract requires a client that cannot check locally to send +// anyway and let the Exchange decide. +func (r *WellKnownRequirementsReader) ResolveRegistrationRequirements( + ctx context.Context, exchange string, +) (RegistrationRequirements, error) { + if !helpers.IsBareDomain(exchange) { + return RegistrationRequirements{}, fmt.Errorf( + "resolvers: registration requirements: %w: %q is not a bare domain", + helpers.ErrInvalidHost, exchange) + } + if r.allow != nil && !r.allow(exchange) { + return RegistrationRequirements{}, fmt.Errorf("%w: %q", ErrExchangeNotPermitted, exchange) + } + // Bounded whatever client was injected: WellKnownOptions.HTTP admits one with + // no timeout at all, and a caller may hold no deadline of its own. The ceiling + // is the endpoint resolver's, so both manifest fetches wait the same. + fetchCtx, cancel := context.WithTimeout(ctx, maxManifestFetch) + defer cancel() + url := r.scheme + "://" + exchange + "/.well-known/ramp.json" + doc, err := fetchWellKnownDoc(fetchCtx, r.http, url) + if err != nil { + return RegistrationRequirements{}, fmt.Errorf( + "resolvers: registration requirements for %q: %w", exchange, err) + } + if !doc.describesExchange() { + return RegistrationRequirements{}, fmt.Errorf("%w: host=%q", ErrManifestNotExchange, exchange) + } + reqs := RegistrationRequirements{TermsDigest: doc.termsDigest(), Verdict: helpers.SchemaNotPublished} + // The schema is compiled from the bytes AS SERVED, because every cap the rules + // state is defined over those bytes. json.RawMessage is the sub-document the + // decoder saw, so nothing has re-encoded it on the way here. + if raw := doc.registrationSchemaBytes(); len(raw) > 0 { + reqs.Schema, reqs.Verdict = helpers.CompileRegistrationSchema(raw) + } + return reqs, nil +} + +// roleExchange is the proto-JSON rendering of Role.ROLE_EXCHANGE. A manifest may +// also carry the enum's number, which proto-JSON permits, so both are accepted. +const roleExchange = "ROLE_EXCHANGE" + +// roleExchangeNumber is Role.ROLE_EXCHANGE's field number, accepted because +// proto-JSON allows an enum to travel as its number. +const roleExchangeNumber = "2" + +// describesExchange reports whether the manifest names the Exchange role. An +// absent or unparsable role is not an Exchange: the contract makes the field +// required, and reading silence as assent would leave the check advisory. +func (d *wellKnownDoc) describesExchange() bool { + var s string + if err := json.Unmarshal(d.Role, &s); err == nil { + return s == roleExchange + } + var n json.Number + if err := json.Unmarshal(d.Role, &n); err == nil { + return n.String() == roleExchangeNumber + } + return false +} + +// absentMember reports whether a raw manifest member carries no value at all — +// the key was not served, or it was served as JSON null. Both are the contract's +// "absent", and null needs naming because encoding/json decodes it into a string +// without complaint, leaving the zero value behind: a caller that only checked the +// error would read a null digest as a present, empty one. +func absentMember(raw json.RawMessage) bool { + return len(raw) == 0 || string(raw) == "null" +} + +// termsDigest returns the manifest's terms_digest, or nil when the Exchange +// publishes none. +// +// A member carrying anything but a string reads as ABSENT rather than as a fault. +// The manifest is a third party's document and this is one optional member of it, +// so the alternative is failing a document the endpoint and key faces would have +// read fine — see the wellKnownDoc field comment. It is also what both ports +// already do with the same member, so the three agree on the answer rather than +// on an error. +func (d *wellKnownDoc) termsDigest() *string { + if absentMember(d.TermsDigest) { + return nil + } + var s string + if err := json.Unmarshal(d.TermsDigest, &s); err != nil { + return nil + } + return &s +} + +// registrationSchemaBytes returns the published data_schema exactly as served, or +// nil when the block or the member is absent. Absent and blank are the same answer +// to the caller — CompileRegistrationSchema reads either as "publishes none" — but +// the two are distinguished here anyway so a blank member reaches the rule that +// defines blankness rather than a rule invented at this layer. +// +// A block that is not an object reads as absent, for the reason termsDigest gives. +func (d *wellKnownDoc) registrationSchemaBytes() []byte { + if absentMember(d.AccountRegistration) { + return nil + } + var block accountRegistration + if err := json.Unmarshal(d.AccountRegistration, &block); err != nil { + return nil + } + return block.DataSchema +} diff --git a/sdk/go/resolvers/registrationrequirements_test.go b/sdk/go/resolvers/registrationrequirements_test.go new file mode 100644 index 00000000..05e2cf2c --- /dev/null +++ b/sdk/go/resolvers/registrationrequirements_test.go @@ -0,0 +1,253 @@ +package resolvers_test + +import ( + "context" + "encoding/json" + "errors" + "net/http" + "net/http/httptest" + "testing" + + "github.com/RAMP-Protocol/protocol/sdk/go/helpers" + "github.com/RAMP-Protocol/protocol/sdk/go/resolvers" +) + +// requirementsManifest serves a manifest carrying the members the registration +// reader looks at. A nil digest or schema omits the member entirely, which is how +// "this Exchange publishes none" looks on the wire. It counts requests, because +// the reader's central promise is that every read is a fresh fetch. +func requirementsManifest(role string, digest *string, schema json.RawMessage, hits *int) http.Handler { + return http.HandlerFunc(func(w http.ResponseWriter, _ *http.Request) { + if hits != nil { + *hits++ + } + doc := map[string]any{"role": role, "endpoint": "https://exchange.test"} + if digest != nil { + doc["terms_digest"] = *digest + } + if schema != nil { + doc["account_registration"] = map[string]any{"data_schema": schema} + } + _ = json.NewEncoder(w).Encode(doc) + }) +} + +// loopbackReader builds a reader against a loopback manifest server. The default +// client is SSRF-guarded and would refuse the address, so the test injects an +// unguarded one exactly as the endpoint-resolver tests do. +func loopbackReader(allow func(string) bool) *resolvers.WellKnownRequirementsReader { + return resolvers.NewWellKnownRequirementsReader(resolvers.WellKnownOptions{ + Scheme: "http", + HTTP: http.DefaultClient, + Allow: allow, + }) +} + +const acceptedSchema = `{"type":"object","required":["legal_entity"],` + + `"properties":{"legal_entity":{"type":"string"}}}` + +func TestRequirements_readsDigestAndSchema(t *testing.T) { + digest := "sha256:" + repeat64("ab") + srv := httptest.NewServer(requirementsManifest( + "ROLE_EXCHANGE", &digest, json.RawMessage(acceptedSchema), nil)) + defer srv.Close() + + got, err := loopbackReader(nil).ResolveRegistrationRequirements( + context.Background(), hostOf(t, srv)) + if err != nil { + t.Fatalf("resolve requirements: %v", err) + } + if got.TermsDigest == nil || *got.TermsDigest != digest { + t.Fatalf("terms digest = %v, want %q", got.TermsDigest, digest) + } + if got.Verdict != helpers.SchemaAccepted { + t.Fatalf("verdict = %v, want accepted", got.Verdict) + } + // The schema is not merely present, it is the one that was served: a payload + // missing the required member must fail against it. + if fails := got.Schema.Validate(map[string]any{}); len(fails) == 0 { + t.Fatal("compiled schema accepted a payload missing a required member") + } + if fails := got.Schema.Validate(map[string]any{"legal_entity": "Acme"}); len(fails) != 0 { + t.Fatalf("compiled schema refused a conforming payload: %v", fails) + } +} + +// The rule this reader exists for: a cached endpoint is fine, a cached digest is +// not. Two reads must produce two fetches, and the second must see the new value. +func TestRequirements_everyReadIsAFreshFetch(t *testing.T) { + digest := "sha256:" + repeat64("ab") + hits := 0 + srv := httptest.NewServer(requirementsManifest("ROLE_EXCHANGE", &digest, nil, &hits)) + defer srv.Close() + r := loopbackReader(nil) + host := hostOf(t, srv) + + if _, err := r.ResolveRegistrationRequirements(context.Background(), host); err != nil { + t.Fatalf("first read: %v", err) + } + digest = "sha256:" + repeat64("cd") + got, err := r.ResolveRegistrationRequirements(context.Background(), host) + if err != nil { + t.Fatalf("second read: %v", err) + } + if hits != 2 { + t.Fatalf("manifest fetches = %d, want 2; the digest was served from a cache", hits) + } + if got.TermsDigest == nil || *got.TermsDigest != digest { + t.Fatalf("second read returned %v, want the revised %q", got.TermsDigest, digest) + } +} + +func TestRequirements_publishingNeitherIsANormalAnswer(t *testing.T) { + srv := httptest.NewServer(requirementsManifest("ROLE_EXCHANGE", nil, nil, nil)) + defer srv.Close() + + got, err := loopbackReader(nil).ResolveRegistrationRequirements( + context.Background(), hostOf(t, srv)) + if err != nil { + t.Fatalf("an Exchange publishing neither member is not an error: %v", err) + } + if got.TermsDigest != nil { + t.Fatalf("terms digest = %q, want nil", *got.TermsDigest) + } + if got.Schema != nil || got.Verdict != helpers.SchemaNotPublished { + t.Fatalf("schema = %v verdict = %v, want nil / not_published", got.Schema, got.Verdict) + } +} + +// A schema this SDK refuses is reported, never raised: the contract requires a +// client that cannot check locally to send anyway and let the Exchange decide. +func TestRequirements_aRefusedSchemaIsAVerdictNotAnError(t *testing.T) { + wrongDialect := json.RawMessage( + `{"$schema":"https://json-schema.org/draft/2019-09/schema","type":"object"}`) + srv := httptest.NewServer(requirementsManifest("ROLE_EXCHANGE", nil, wrongDialect, nil)) + defer srv.Close() + + got, err := loopbackReader(nil).ResolveRegistrationRequirements( + context.Background(), hostOf(t, srv)) + if err != nil { + t.Fatalf("a refused schema must not fail the read: %v", err) + } + if got.Verdict != helpers.SchemaWrongDialect { + t.Fatalf("verdict = %v, want wrong_dialect", got.Verdict) + } + if got.Schema != nil { + t.Fatal("a refused schema must not be returned") + } + // And the nil schema must behave as "nothing to enforce" rather than as a veto. + if fails := got.Schema.Validate(map[string]any{}); len(fails) != 0 { + t.Fatalf("a refused schema became a local veto: %v", fails) + } +} + +func TestRequirements_refusesBeforeDialling(t *testing.T) { + hits := 0 + srv := httptest.NewServer(requirementsManifest("ROLE_EXCHANGE", nil, nil, &hits)) + defer srv.Close() + host := hostOf(t, srv) + + t.Run("a value that is not a bare domain", func(t *testing.T) { + // A path smuggled into the domain would choose WHAT is fetched, because the + // URL is built by concatenation. + _, err := loopbackReader(nil).ResolveRegistrationRequirements( + context.Background(), host+"/evil") + if !errors.Is(err, helpers.ErrInvalidHost) { + t.Fatalf("err = %v, want ErrInvalidHost", err) + } + }) + + t.Run("a domain the deployment excludes", func(t *testing.T) { + _, err := loopbackReader(func(string) bool { return false }). + ResolveRegistrationRequirements(context.Background(), host) + if !errors.Is(err, resolvers.ErrExchangeNotPermitted) { + t.Fatalf("err = %v, want ErrExchangeNotPermitted", err) + } + }) + + if hits != 0 { + t.Fatalf("manifest fetches = %d, want 0; a refusal reached the network", hits) + } +} + +// Registration requirements are an Exchange's to publish. A manifest claiming +// another role is refused rather than read for members it has no business +// carrying — and so is one naming no role at all, since the contract requires the +// field and reading silence as assent would leave the check advisory. +func TestRequirements_refusesAManifestThatIsNotAnExchange(t *testing.T) { + for _, role := range []string{"ROLE_BROKER", "ROLE_AGENT", "ROLE_PUBLISHER", ""} { + name := role + if name == "" { + name = "no role at all" + } + t.Run(name, func(t *testing.T) { + srv := httptest.NewServer(requirementsManifest(role, nil, nil, nil)) + defer srv.Close() + _, err := loopbackReader(nil).ResolveRegistrationRequirements( + context.Background(), hostOf(t, srv)) + if !errors.Is(err, resolvers.ErrManifestNotExchange) { + t.Fatalf("err = %v, want ErrManifestNotExchange", err) + } + }) + } +} + +// The sentinel this reader never raises, asserted rather than assumed. +// +// ErrManifestUnusable exists for a reader STRICTER than this one. The two ways a +// manifest disappoints this one are both deliberate non-errors, and each is a +// decision another test would not notice being reversed: an off-spec member reads +// as absent so one of them cannot fail a document the endpoint and key faces would +// have read, and a document that does not decode is a transport failure because the +// next fetch may decode. Raising the verdict for either would make this reader +// refuse finally where it currently tolerates or retries. +func TestRequirements_neverRaisesTheUnusableVerdictItself(t *testing.T) { + for _, tc := range []struct { + name string + body string + }{ + // Both optional members carrying a type the contract does not admit. + {"off-spec members", `{"role":"ROLE_EXCHANGE","terms_digest":7,` + + `"account_registration":"not an object"}`}, + {"not JSON at all", `502 Bad Gateway`}, + {"truncated mid-document", `{"role":"ROLE_EXCHANGE","terms_`}, + } { + t.Run(tc.name, func(t *testing.T) { + srv := httptest.NewServer(http.HandlerFunc( + func(w http.ResponseWriter, _ *http.Request) { + _, _ = w.Write([]byte(tc.body)) + })) + defer srv.Close() + + _, err := loopbackReader(nil).ResolveRegistrationRequirements( + context.Background(), hostOf(t, srv)) + if errors.Is(err, resolvers.ErrManifestUnusable) { + t.Fatalf("err = %v carries ErrManifestUnusable; this reader tolerates "+ + "an off-spec member and retries an undecodable document", err) + } + }) + } +} + +// proto-JSON lets an enum travel as its number, so a manifest that spells the +// role that way is the same manifest. +func TestRequirements_acceptsTheRoleAsANumber(t *testing.T) { + srv := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, _ *http.Request) { + _, _ = w.Write([]byte(`{"role":2,"endpoint":"https://exchange.test"}`)) + })) + defer srv.Close() + + if _, err := loopbackReader(nil).ResolveRegistrationRequirements( + context.Background(), hostOf(t, srv)); err != nil { + t.Fatalf("numeric role refused: %v", err) + } +} + +// repeat64 builds a 64-character hex body for a sha256 digest from a 2-char unit. +func repeat64(unit string) string { + out := "" + for len(out) < 64 { + out += unit + } + return out[:64] +} diff --git a/sdk/go/resolvers/transportdefaults_test.go b/sdk/go/resolvers/transportdefaults_test.go new file mode 100644 index 00000000..fd6b0c1e --- /dev/null +++ b/sdk/go/resolvers/transportdefaults_test.go @@ -0,0 +1,141 @@ +package resolvers_test + +import ( + "context" + "crypto/ed25519" + "encoding/base64" + "fmt" + "net/http" + "net/http/httptest" + "testing" + "time" + + "github.com/RAMP-Protocol/protocol/sdk/go/resolvers" +) + +// Which transport each resolver takes when the caller injects none. +// +// # Why this is a test and not a sentence +// +// The rule is that a resolver's default follows its URL's PROVENANCE: a fixed, +// operator-chosen address takes the plain client, a host another party named +// takes the guarded one. That rule is written down in three places — the +// WellKnownOptions.HTTP comment, docs/design-history.md, and the published +// threat model, which states it per language in a table. +// +// It has now drifted from the code twice. The scheme guard was added to the +// dial and not walked back through its callers, so a doc sentence beginning +// "every fetch" stayed standing after it stopped being true; and the two ports +// mirrored this package's options struct without mirroring the rule, then +// explained the result in a docstring that had also stopped being true. Prose +// cannot detect either. A socket can. +// +// SO: CHANGING ANY ROW BELOW OBLIGES AN EDIT TO THE PUBLISHED THREAT MODEL +// (website/src/content/docs/security/threat-model.mdx), which names these +// postures per language. That is the whole point of the gate — the failure is +// the reminder. +// +// # What is not here +// +// The WBA directory resolver's guarded default is pinned already, by +// wbakeyresolver_ssrf_internal_test.go and its two port mirrors; its guard is +// additionally not opt-outable, which those tests cover and this one would not. +// Asserting it a second time here would be one behaviour with two owners. +// +// This is deliberately NOT a shared corpus. A *-vectors.json has to be replayed +// by all three languages against the same expectations, and the expectations +// here legitimately DIFFER per language — that difference is the subject. The +// ports carry their own copy of this table for the same reason. + +// dialObserved reports whether a resolver reached the loopback origin. A guarded +// default refuses the reserved address before the request leaves the process, so +// "the handler ran" is the observable that separates the two transports. +func dialObserved(t *testing.T, exercise func(host string) error) bool { + t.Helper() + reached := false + srv := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) { + reached = true + w.Header().Set("Content-Type", "application/json") + // Anchored to the host that served it, so the endpoint face's own + // same-host rule cannot be what refuses. + _, _ = fmt.Fprintf(w, `{"role":"ROLE_EXCHANGE","endpoint":"https://%s","keys":[%s]}`, + r.Host, testJWK(t)) + })) + defer srv.Close() + // The error is deliberately ignored: a guarded refusal and a happy answer are + // both expected outcomes here, and which one occurred is what `reached` says. + _ = exercise(hostOf(t, srv)) + return reached +} + +func testJWK(t *testing.T) string { + t.Helper() + pub, _, err := ed25519.GenerateKey(nil) + if err != nil { + t.Fatalf("generate key: %v", err) + } + return fmt.Sprintf(`{"kid":"ex.v1","kty":"OKP","crv":"Ed25519","x":%q}`, + base64.RawURLEncoding.EncodeToString(pub)) +} + +func TestResolverTransportDefaults(t *testing.T) { + for _, tc := range []struct { + name string + // reaches is TRUE where the default is the plain client. + reaches bool + why string + exercise func(host string) error + }{ + { + name: "key resolver dials plain", + reaches: true, + why: "its URL is a fixed, operator-chosen JWKS address — nobody else can " + + "point it, and an on-prem directory may legitimately be private", + exercise: func(host string) error { + r := resolvers.NewWellKnownKeyResolver( + "http://"+host+"/.well-known/ramp.json", resolvers.WellKnownOptions{TTL: time.Hour}) + _, err := r.Resolve(context.Background(), "ex.v1") + return err + }, + }, + { + name: "endpoint resolver dials guarded", + reaches: false, + why: "its host is an Offer.exchange domain, so another party chose the " + + "address. The two ports do NOT match this row yet — see their copies", + exercise: func(host string) error { + r := resolvers.NewWellKnownEndpointResolver(resolvers.WellKnownOptions{ + TTL: time.Hour, Scheme: "http", + }) + _, err := r.ResolveEndpoint(context.Background(), host) + return err + }, + }, + { + name: "requirements reader dials guarded", + reaches: false, + why: "its host is a RegisterRequest.exchange domain, named by the caller per call", + exercise: func(host string) error { + r := resolvers.NewWellKnownRequirementsReader(resolvers.WellKnownOptions{Scheme: "http"}) + _, err := r.ResolveRegistrationRequirements(context.Background(), host) + return err + }, + }, + } { + t.Run(tc.name, func(t *testing.T) { + // Both guards are env-driven, so the flags are cleared rather than + // inherited from whatever shell ran the suite: without this a developer + // with SKIP_SSRF set sees the guarded rows fail for a reason that has + // nothing to do with the defaults. t.Setenv restores them after the test. + t.Setenv("SKIP_SSRF", "") + t.Setenv("ALLOW_INSECURE", "") + if got := dialObserved(t, tc.exercise); got != tc.reaches { + verb := map[bool]string{true: "reached", false: "refused"} + t.Fatalf("default transport %s the loopback origin, want %s — %s.\n"+ + "If this change is intended, the per-language table in the published "+ + "threat model has to change with it.", + verb[got], verb[tc.reaches], tc.why) + } + }) + } +} diff --git a/sdk/go/resolvers/wellknowndoc_shared_test.go b/sdk/go/resolvers/wellknowndoc_shared_test.go new file mode 100644 index 00000000..825df2ca --- /dev/null +++ b/sdk/go/resolvers/wellknowndoc_shared_test.go @@ -0,0 +1,165 @@ +package resolvers_test + +import ( + "context" + "crypto/ed25519" + "encoding/base64" + "fmt" + "net/http" + "net/http/httptest" + "testing" + "time" + + "github.com/RAMP-Protocol/protocol/sdk/go/helpers" + "github.com/RAMP-Protocol/protocol/sdk/go/resolvers" +) + +// One fetch decodes one manifest for three faces — keys, endpoint and registration +// requirements — so a member only one of them reads must never be able to fail the +// document for the other two. +// +// The manifest is a THIRD PARTY's, and terms_digest and account_registration are +// optional members of it. An Exchange that publishes one of them with the wrong +// type is off-spec, but the answer to that has to be "this face sees nothing" +// rather than "nobody can read this document": a decode failure is classified as a +// transport failure, which is RETRYABLE, so failing the document would turn every +// usage report and every key resolution against that Exchange into an endless +// retry over a member neither of them reads. Usage reports are how money moves. +// +// The wrong-type answer is also what both ports already give, so the three SDKs +// agree on a value rather than on an error. + +// literalManifest serves body verbatim, which is what lets these tests put a +// member on the wire that no typed encoder here would produce. +func literalManifest(body string, hits *int) http.Handler { + return http.HandlerFunc(func(w http.ResponseWriter, _ *http.Request) { + if hits != nil { + *hits++ + } + w.Header().Set("Content-Type", "application/json") + _, _ = w.Write([]byte(body)) + }) +} + +// offSpecMembers are optional members served with a type the contract does not +// admit, plus the two JSON nulls. Null is listed because it is the case a type +// check alone misses: encoding/json decodes null into a string without error and +// leaves the zero value, so a reader that only consulted the error would report a +// present, empty digest where the ports report none. +var offSpecMembers = []struct { + name string + member string +}{ + {"digest is a number", `"terms_digest":123`}, + {"digest is a bool", `"terms_digest":false`}, + {"digest is an object", `"terms_digest":{"sha256":"x"}`}, + {"digest is null", `"terms_digest":null`}, + {"registration is a string", `"account_registration":"https://exchange.test/signup"`}, + {"registration is an array", `"account_registration":[1,2]`}, + {"registration is null", `"account_registration":null`}, +} + +func TestWellKnownDoc_anOffSpecOptionalMemberDoesNotFailTheEndpointFace(t *testing.T) { + for _, tc := range offSpecMembers { + t.Run(tc.name, func(t *testing.T) { + // The advertised endpoint must anchor to the host that served the + // manifest, so the body is built per request from the Host header + // rather than from a value fixed before the listener has a port. + srv := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, req *http.Request) { + w.Header().Set("Content-Type", "application/json") + _, _ = fmt.Fprintf(w, `{"ver":%q,"role":"ROLE_EXCHANGE","endpoint":"https://%s",%s}`, + helpers.WellKnownManifestVersion, req.Host, tc.member) + })) + defer srv.Close() + host := hostOf(t, srv) + + r := resolvers.NewWellKnownEndpointResolver(resolvers.WellKnownOptions{ + TTL: time.Hour, Scheme: "http", HTTP: http.DefaultClient, + }) + got, err := r.ResolveEndpoint(context.Background(), host) + if err != nil { + t.Fatalf("resolve endpoint: %v", err) + } + if want := "https://" + host; got != want { + t.Errorf("endpoint = %q, want %q", got, want) + } + }) + } +} + +func TestWellKnownDoc_anOffSpecOptionalMemberDoesNotFailTheKeyFace(t *testing.T) { + pub, _, err := ed25519.GenerateKey(nil) + if err != nil { + t.Fatalf("generate key: %v", err) + } + x := base64.RawURLEncoding.EncodeToString(pub) + + for _, tc := range offSpecMembers { + t.Run(tc.name, func(t *testing.T) { + srv := httptest.NewServer(literalManifest(fmt.Sprintf( + `{"ver":%q,"role":"ROLE_EXCHANGE","keys":[{"kid":"ex.v1","kty":"OKP","crv":"Ed25519","x":%q}],%s}`, + helpers.WellKnownManifestVersion, x, tc.member), nil)) + defer srv.Close() + + r := resolvers.NewWellKnownKeyResolver(srv.URL, resolvers.WellKnownOptions{TTL: time.Hour}) + got, err := r.Resolve(context.Background(), "ex.v1") + if err != nil { + t.Fatalf("resolve key: %v", err) + } + if !got.Equal(pub) { + t.Error("resolved key does not match the served one") + } + }) + } +} + +func TestWellKnownDoc_anOffSpecOptionalMemberReadsAsAbsent(t *testing.T) { + for _, tc := range offSpecMembers { + t.Run(tc.name, func(t *testing.T) { + srv := httptest.NewServer(literalManifest(fmt.Sprintf( + `{"ver":%q,"role":"ROLE_EXCHANGE","endpoint":"https://exchange.test",%s}`, + helpers.WellKnownManifestVersion, tc.member), nil)) + defer srv.Close() + + got, err := loopbackReader(nil).ResolveRegistrationRequirements( + context.Background(), hostOf(t, srv)) + if err != nil { + t.Fatalf("resolve requirements: %v", err) + } + if got.TermsDigest != nil { + t.Errorf("terms digest = %q, want absent — a member of the wrong type is not a value", + *got.TermsDigest) + } + if got.Schema != nil { + t.Error("schema is set; an unreadable account_registration block publishes none") + } + if got.Verdict != helpers.SchemaNotPublished { + t.Errorf("verdict = %v, want %v", got.Verdict, helpers.SchemaNotPublished) + } + }) + } +} + +// A conformant manifest still reads exactly as before. The tolerance above is for +// members this SDK cannot use, and it must not become tolerance for members it can: +// a good digest is still returned, and a good schema still compiles. +func TestWellKnownDoc_toleranceDoesNotSwallowAConformantMember(t *testing.T) { + digest := "sha256:" + repeat64("cd") + srv := httptest.NewServer(literalManifest(fmt.Sprintf( + `{"role":"ROLE_EXCHANGE","endpoint":"https://exchange.test","terms_digest":%q,`+ + `"account_registration":{"data_schema":%s}}`, digest, acceptedSchema), nil)) + defer srv.Close() + + got, err := loopbackReader(nil).ResolveRegistrationRequirements( + context.Background(), hostOf(t, srv)) + if err != nil { + t.Fatalf("resolve requirements: %v", err) + } + if got.TermsDigest == nil || *got.TermsDigest != digest { + t.Errorf("terms digest = %v, want %q", got.TermsDigest, digest) + } + if got.Verdict != helpers.SchemaAccepted || got.Schema == nil { + t.Errorf("verdict = %v, schema nil = %t; want an accepted, usable schema", + got.Verdict, got.Schema == nil) + } +} diff --git a/sdk/parity/symbol-map.json b/sdk/parity/symbol-map.json index 561130d8..3d3f92d5 100644 --- a/sdk/parity/symbol-map.json +++ b/sdk/parity/symbol-map.json @@ -26,6 +26,7 @@ "connect.WithMaxContentBytes": "Go functional-option builder; py/ts pass options via kwargs/options objects.", "connect.WithOfferKey": "Go functional-option builder; py/ts pass options via kwargs/options objects.", "connect.WithProofWindow": "Go functional-option builder; py/ts pass options via kwargs/options objects.", + "connect.WithRegistrationRequirements": "Go functional-option builder; py/ts pass options via kwargs/options objects.", "connect.WithRequestIDFunc": "Go functional-option builder; py/ts pass options via kwargs/options objects.", "connect.WithRequester": "Go functional-option builder; py/ts pass options via kwargs/options objects.", "connect.WithSignWindow": "Go functional-option builder; py/ts pass options via kwargs/options objects.", @@ -88,7 +89,6 @@ "helpers.ComponentParam": "Go value type for an RFC 9421 covered-component parameter; py/ts model components inline.", "helpers.CoveredComponent": "Go value type for an RFC 9421 covered component; py/ts model components inline.", "helpers.ErrAcceptanceSignatureInvalid": "Go errors.Is sentinel; py/ts express verification failures via typed failure unions / exception classes, not per-reason named sentinels.", - "helpers.ErrRequestAcceptanceSignatureInvalid": "Go errors.Is sentinel; py/ts return false for a request-acceptance mismatch rather than exporting a sentinel.", "helpers.ErrAudienceIdentity": "Go errors.Is sentinel for an unusable configured Exchange identity; py/ts raise/throw instead of exporting sentinels.", "helpers.ErrBrokenSignatureChain": "Go errors.Is sentinel; py/ts express verification failures via typed failure unions / exception classes, not per-reason named sentinels.", "helpers.ErrDigestMismatch": "Go errors.Is sentinel; py/ts express verification failures via typed failure unions / exception classes, not per-reason named sentinels.", @@ -111,6 +111,7 @@ "helpers.ErrOfferExpired": "Go errors.Is sentinel; py/ts express verification failures via typed failure unions / exception classes, not per-reason named sentinels.", "helpers.ErrOfferSignatureInvalid": "Go errors.Is sentinel; py/ts express verification failures via typed failure unions / exception classes, not per-reason named sentinels.", "helpers.ErrProofOfPossessionMismatch": "Go errors.Is sentinel; py/ts express verification failures via typed failure unions / exception classes, not per-reason named sentinels.", + "helpers.ErrRequestAcceptanceSignatureInvalid": "Go errors.Is sentinel; py/ts return false for a request-acceptance mismatch rather than exporting a sentinel.", "helpers.ErrSignatureLifetimeTooLong": "Go errors.Is sentinel; py/ts express verification failures via typed failure unions / exception classes, not per-reason named sentinels.", "helpers.ErrSignatureVerify": "Go errors.Is sentinel; py/ts express verification failures via typed failure unions / exception classes, not per-reason named sentinels.", "helpers.ErrTooManyHops": "Go errors.Is sentinel; py/ts express verification failures via typed failure unions / exception classes, not per-reason named sentinels.", @@ -136,7 +137,7 @@ "helpers.RegistrationDataUncanonicalizable": "Member of the mapped helpers.RegistrationDataVerdict vocabulary. Python and TypeScript spell it as a literal; the shared registration-schema corpus pins the token.", "helpers.RegistrationSchemaCompileTimeout": "Go-only wall-clock backstop on compilation, not part of the accepted/refused contract and deliberately absent from the ports: Go's runtime preempts, while a CPU-bound spin holds CPython's interpreter and blocks Node's event loop, so a timer there cannot interrupt the work it names. What bounds all three identically is static — the size, depth and evaluation caps and the pattern alphabet — and no admitted schema should ever reach this timeout.", "helpers.RequestAcceptancePayload": "Go typed-protobuf builder for the request-acceptance payload; Python and TypeScript build their language-native payload objects inside the mapped canonicalizer/client.", - "helpers.RetrievalAuthFailureReasonFromToken": "Go lookup from the delivery edge's refusal token to the typed enum; py/ts branch on the token string directly.", + "helpers.RetrievalAuthFailureReasonFromToken": "Go exposes the delivery edge's refusal-token table as an L1 lookup, because the Exchange and the client both read it; in the ports the content leg is its only consumer, so each keeps the SAME table private to that module. The table is the protocol's own record — ramp.proto names the token beside each enum value — never derived from the enum's spelling, and synthesized-detail-vectors.json replays every recorded token in all three languages so the three copies cannot come apart again.", "helpers.SchemaAccepted": "Member of the mapped helpers.SchemaVerdict vocabulary. Python and TypeScript spell it as a literal; the shared registration-schema corpus pins the token.", "helpers.SchemaCompileTimeout": "Member of the mapped helpers.SchemaVerdict vocabulary. Python and TypeScript spell it as a literal; the shared registration-schema corpus pins the token.", "helpers.SchemaMalformed": "Member of the mapped helpers.SchemaVerdict vocabulary. Python and TypeScript spell it as a literal; the shared registration-schema corpus pins the token.", @@ -153,8 +154,8 @@ "helpers.SchemaWrongDialect": "Member of the mapped helpers.SchemaVerdict vocabulary. Python and TypeScript spell it as a literal; the shared registration-schema corpus pins the token.", "helpers.SharedValidator": "Go protovalidate validator singleton; TS/Python ship no protovalidate face.", "helpers.SignOfferAcceptanceWith": "Go Signer-custody variant of SignOfferAcceptance so the SDK never holds the key; py/ts pass key material directly to their single acceptance signer.", - "helpers.SignRequestAcceptanceWith": "Go Signer-custody variant of SignRequestAcceptance; Python custody is a SigningTransport method and TypeScript passes its CryptoKey directly.", "helpers.SignOptions": "Go options struct for SignRequest; py/ts pass options via kwargs/options objects.", + "helpers.SignRequestAcceptanceWith": "Go Signer-custody variant of SignRequestAcceptance; Python custody is a SigningTransport method and TypeScript passes its CryptoKey directly.", "helpers.SignatureAgentFromContext": "Go context.Context accessor; py/ts thread signature-agent state explicitly.", "helpers.SignedURL": "Go signed-URL result value type; py/ts return language-native result objects.", "helpers.Signer": "Go Signer interface; py/ts inject a sign function (TS Ed25519SignFn; Python a signing callable) rather than a named Signer type — divergent handle shape, not a missing operation.", @@ -246,6 +247,11 @@ "python": null, "ts": "createClient" }, + "connect.RegistrationRequirementsReader": { + "allowlist_reason": null, + "python": "RegistrationRequirementsReader", + "ts": "RegistrationRequirementsReader" + }, "connect.Validation": { "allowlist_reason": null, "python": "Validation", @@ -379,16 +385,16 @@ "python": "jcs_acceptance_payload", "ts": "acceptancePayload" }, - "helpers.CanonicalRequestAcceptanceBytes": { - "allowlist_reason": null, - "python": "jcs_request_acceptance_payload", - "ts": "requestAcceptancePayload" - }, "helpers.CanonicalOfferBytes": { "allowlist_reason": null, "python": "canonical_offer_payload", "ts": "canonicalOfferPayload" }, + "helpers.CanonicalRequestAcceptanceBytes": { + "allowlist_reason": null, + "python": "jcs_request_acceptance_payload", + "ts": "requestAcceptancePayload" + }, "helpers.CanonicalRestrictionToken": { "allowlist_reason": null, "python": "canonical_restriction_token", @@ -714,16 +720,16 @@ "python": "sign_offer_acceptance_jcs", "ts": "signOfferAcceptance" }, - "helpers.SignRequestAcceptance": { - "allowlist_reason": null, - "python": "sign_request_acceptance_jcs", - "ts": "signRequestAcceptance" - }, "helpers.SignRequest": { "allowlist_reason": null, "python": "sign_request", "ts": "signRequest" }, + "helpers.SignRequestAcceptance": { + "allowlist_reason": null, + "python": "sign_request_acceptance_jcs", + "ts": "signRequestAcceptance" + }, "helpers.SignURLEd25519": { "allowlist_reason": null, "python": "sign_ed25519_signed_url", @@ -784,16 +790,16 @@ "python": "verify_offer_acceptance_jcs", "ts": "verifyOfferAcceptance" }, - "helpers.VerifyRequestAcceptance": { - "allowlist_reason": null, - "python": "verify_request_acceptance_jcs", - "ts": "verifyRequestAcceptance" - }, "helpers.VerifyRequest": { "allowlist_reason": null, "python": "verify_request", "ts": "verifyRequestServer" }, + "helpers.VerifyRequestAcceptance": { + "allowlist_reason": null, + "python": "verify_request_acceptance_jcs", + "ts": "verifyRequestAcceptance" + }, "helpers.VerifyURLEd25519": { "allowlist_reason": null, "python": "verify_ed25519_signed_url", @@ -854,6 +860,11 @@ "python": "EndpointRefusedError", "ts": "EndpointRefused" }, + "resolvers.ErrExchangeNotPermitted": { + "allowlist_reason": null, + "python": "ExchangeNotPermittedError", + "ts": "ExchangeNotPermitted" + }, "resolvers.ErrKeyExpired": { "allowlist_reason": null, "python": "KeyExpiredError", @@ -864,6 +875,16 @@ "python": "KeyRevokedError", "ts": "KeyRevoked" }, + "resolvers.ErrManifestNotExchange": { + "allowlist_reason": null, + "python": "ManifestNotExchangeError", + "ts": "ManifestNotExchange" + }, + "resolvers.ErrManifestUnusable": { + "allowlist_reason": null, + "python": "ManifestUnusableError", + "ts": "ManifestUnusable" + }, "resolvers.ErrManifestVersionRefused": { "allowlist_reason": null, "python": "ManifestVersionRefusedError", @@ -914,11 +935,21 @@ "python": null, "ts": "createWellKnownKeyResolver" }, + "resolvers.NewWellKnownRequirementsReader": { + "allowlist_reason": "Go NewWellKnownRequirementsReader factory folds into the Python class constructor (WellKnownRequirementsReader(...)); idiomatic Python exposes the class, not a separate factory symbol. TS keeps the createWellKnownRequirementsReader factory.", + "python": null, + "ts": "createWellKnownRequirementsReader" + }, "resolvers.OfferDirectoryFetcher": { "allowlist_reason": null, "python": "DirectoryFetch", "ts": "OfferDirectoryFetch" }, + "resolvers.RegistrationRequirements": { + "allowlist_reason": null, + "python": "RegistrationRequirements", + "ts": "RegistrationRequirements" + }, "resolvers.SSRFGuard": { "allowlist_reason": null, "python": "ssrf_guard", @@ -958,6 +989,11 @@ "allowlist_reason": "Go options struct (shared by the well-known key/endpoint resolvers) / TS options object folds into Python constructor keyword arguments (WellKnownKeyResolver(...) / WellKnownEndpointResolver(...)); idiomatic Python has no separate options type.", "python": null, "ts": "WellKnownOptions" + }, + "resolvers.WellKnownRequirementsReader": { + "allowlist_reason": null, + "python": "WellKnownRequirementsReader", + "ts": "WellKnownRequirementsReader" } } } diff --git a/sdk/python/ramp_sdk/_hostref.py b/sdk/python/ramp_sdk/_hostref.py index 2f0278dd..556e5a66 100644 --- a/sdk/python/ramp_sdk/_hostref.py +++ b/sdk/python/ramp_sdk/_hostref.py @@ -168,6 +168,28 @@ def _invalid_host(ref: str, why: str) -> ValueError: return ValueError(f"hosts: reference is not a usable host: {why}: {_redact_userinfo(ref)!r}") +_INVALID_HOST_PREFIX = "hosts: reference is not a usable host" + + +def _is_invalid_host_refusal(exc: BaseException) -> bool: + """Whether an exception is the invalid-host refusal :func:`_invalid_host` mints. + + Recognised by its wording rather than its type. The host helpers are L1 and + deliberately raise a bare ``ValueError``; a typed class would cross the + helpers-versus-resolvers sentinel split the SDK documents, so the shared prefix is + what there is to match. The wording is this port's own — the oracle's sentinel + reads ``helpers:`` — which is fine, because what the three languages owe each other + is the verdict and never the sentence. + + It exists because TWO injectable seams can raise it, the endpoint resolver and the + registration-requirements reader, and both have to call it final for the same + reason. Catching the bare type instead would call every other ``ValueError`` final + too — ``json.JSONDecodeError`` is one — and a document a reader could not parse is + a transport failure in the oracle and in the other port. + """ + return isinstance(exc, ValueError) and str(exc).startswith(_INVALID_HOST_PREFIX) + + def _split_components(rest: str) -> tuple[str, str, str]: """Split what follows ``://`` into its authority, its path and its fragment. diff --git a/sdk/python/ramp_sdk/client/__init__.py b/sdk/python/ramp_sdk/client/__init__.py index 29fc62b4..129ccfda 100644 --- a/sdk/python/ramp_sdk/client/__init__.py +++ b/sdk/python/ramp_sdk/client/__init__.py @@ -29,7 +29,7 @@ import httpx -from ramp_sdk.resolvers import _ssrf, guarded_async_client +from ramp_sdk.resolvers import _ssrf, guarded_async_client, guarded_client from ramp_sdk.window import clock_window from . import _verbs @@ -46,7 +46,7 @@ require_dialable_scheme, rpc_headers, ) -from ._verbs import ClientConfig +from ._verbs import ClientConfig, _with_requirements_reader from .content import ( DEFAULT_CONTENT_TIMEOUT_SEC, DEFAULT_MAX_CONTENT_BYTES, @@ -60,13 +60,15 @@ transport_failure, ) from .errors import NOT_CANONICAL_WIRE_NAMING, CallError, CallErrorKind -from .route import EndpointResolver +from .route import EndpointResolver, RegistrationRequirementsReader if TYPE_CHECKING: from wire.models import ( DisputeResponse, + GetAccountStatusResponse, PushResourcesResponse, RefreshCatalogResponse, + RegisterResponse, RemoveResourcesResponse, TransactionResponse, UsageReportResponse, @@ -89,6 +91,7 @@ "ClientConfig", "Content", "EndpointResolver", + "RegistrationRequirementsReader", "Validation", ] @@ -113,7 +116,6 @@ def __init__( http: httpx.AsyncClient | None, guarded: httpx.AsyncClient | None = None, ) -> None: - self._config = config self._owns = http is None self._http = http if http is not None else httpx.AsyncClient( follow_redirects=False, trust_env=False @@ -124,9 +126,15 @@ def __init__( self._guarded = guarded if guarded is not None else ( self._http if not self._owns else guarded_async_client(follow_redirects=False) ) + self._config, self._requirements_http = _with_requirements_reader(config) async def aclose(self) -> None: """Close the transports this client built. An injected one is left alone.""" + # Independent of the RPC legs above: this client is built here whenever the + # caller injected no reader, whether or not it injected an RPC transport, so it + # is closed on its own terms rather than behind that ownership question. + if self._requirements_http is not None: + self._requirements_http.close() if not self._owns: return await self._http.aclose() @@ -248,6 +256,31 @@ async def dispute( status, body = await self._send(plan) return _verbs.finish_dispute(plan, status, body) + async def register(self, request: dict[str, Any]) -> RegisterResponse: + """Create this agent's account at the Exchange the request names. + + Takes no idempotency key: the message carries none, because registering again + returns the same account handle. The plan runs in a thread because it signs and, + when the caller left ``terms_digest`` unset, reads the Exchange's published + requirements from a freshly fetched manifest. + """ + plan = await asyncio.to_thread(_verbs.plan_register, self._config, request) + status, body = await self._send(plan) + return _verbs.finish_register(plan, status, body) + + async def get_account_status( + self, request: dict[str, Any] + ) -> GetAccountStatusResponse: + """Read whether this agent's account at the named Exchange is active. + + An empty ``billing_ref`` in the answer is a NORMAL answer: no account there yet. + """ + plan = await asyncio.to_thread( + _verbs.plan_get_account_status, self._config, request + ) + status, body = await self._send(plan) + return _verbs.finish_get_account_status(plan, status, body) + async def fetch(self, signed_url: str) -> Content: """Retrieve the content a signed delivery URL names, presenting proof of possession of the agent key that URL is bound to. diff --git a/sdk/python/ramp_sdk/client/_call.py b/sdk/python/ramp_sdk/client/_call.py index aae21ba2..010bf52a 100644 --- a/sdk/python/ramp_sdk/client/_call.py +++ b/sdk/python/ramp_sdk/client/_call.py @@ -47,6 +47,7 @@ from pydantic import BaseModel from ramp_sdk.signing_transport import SigningTransport +from ramp_sdk.window import Window #: Caps the response body a single RAMP call will read. #: @@ -81,6 +82,7 @@ def prepare( *, signer: SigningTransport | None, request_id: Callable[[], str] | None, + sign_window: Window | None = None, ) -> tuple[bytes, dict[str, str]]: """Render one request to the bytes that are both signed and sent. @@ -104,7 +106,7 @@ def prepare( if signer is not None: try: signed = signer.sign_outbound( - method="POST", url=url, body=body, authorization="" + method="POST", url=url, body=body, authorization="", window=sign_window ) except Exception as exc: # custody can fail any way it likes # NOT_SIGNABLE, matching what the content leg answers for the same missing @@ -279,6 +281,7 @@ def _connect_envelope_error(op: str, status: int, payload: Any) -> CallError: if not code: code = connect_code_from_status(status) message = envelope.get("message") + detail = error_detail_from(envelope) return CallError( kind_of_connect_code(code), op, @@ -286,7 +289,11 @@ def _connect_envelope_error(op: str, status: int, payload: Any) -> CallError: # The peer's own token, which is the Connect code here. A caller that wants more # than the class reads the typed detail. reason=code or None, - detail=error_detail_from(envelope), + detail=detail, + # The one site that fills peer_message, because this is the one site holding a + # detail the PEER emitted. Every other CallError leaves it empty — including the + # content leg, whose detail this SDK writes itself. + peer_message=detail.message if detail is not None and detail.message else "", cause=message if isinstance(message, str) else None, ) diff --git a/sdk/python/ramp_sdk/client/_verbs.py b/sdk/python/ramp_sdk/client/_verbs.py index 245ed161..6ee75e55 100644 --- a/sdk/python/ramp_sdk/client/_verbs.py +++ b/sdk/python/ramp_sdk/client/_verbs.py @@ -12,7 +12,7 @@ from __future__ import annotations import copy -from dataclasses import dataclass, field +from dataclasses import dataclass, field, replace from typing import TYPE_CHECKING, Any from wire.models import ( @@ -20,10 +20,15 @@ DiscoveryResponse, DisputeRequest, DisputeResponse, + GetAccountStatusRequest, + GetAccountStatusResponse, PushResourcesRequest, PushResourcesResponse, RefreshCatalogRequest, RefreshCatalogResponse, + RegisterRequest, + RegisterResponse, + RegistrationFailureReason, RemoveResourcesRequest, RemoveResourcesResponse, ResourceQuery, @@ -42,7 +47,18 @@ VerifiedOffer, Verifier, ) +from ramp_sdk.errordetail import registration_failure_detail from ramp_sdk.idempotency import generate_idempotency_key +from ramp_sdk.regschema import check_registration_data +from ramp_sdk._hostref import _is_invalid_host_refusal +from ramp_sdk.resolvers import ( + ExchangeNotPermittedError, + ManifestNotExchangeError, + ManifestUnusableError, + WellKnownRequirementsReader, + guarded_client, +) +from ramp_sdk.window import Window from ramp_sdk.wire import ProtocolVersion from ramp_sdk.wire_canon import from_wire_offer @@ -59,11 +75,17 @@ from .._hostref import _redact_userinfo as redact_userinfo from ..hosts import is_bare_domain from .errors import CallError, CallErrorKind, malformed, not_sent -from .route import EndpointResolver, vet_exchange_endpoint +from .route import ( + EndpointResolver, + RegistrationRequirementsReader, + vet_exchange_endpoint, +) if TYPE_CHECKING: from collections.abc import Callable + import httpx + from ramp_sdk.signing_transport import SigningTransport from ramp_sdk.window import Window @@ -92,6 +114,25 @@ class ClientConfig: #: Turns an offer's exchange domain into that Exchange's own advertised origin. Never #: configuration — a usage report and a dispute go where the signed offer says. endpoint_resolver: EndpointResolver | None = None + #: Reports what one Exchange asks of a registration — the terms revision submitting + #: one accepts, and the schema its registration_data must match. The reader it takes + #: holds no document cache, and that is the point rather than an implementation + #: detail: the contract requires a registering client to read the terms digest from a + #: FRESHLY fetched manifest, so an implementation serving it from a cache breaks the + #: rule the field exists to record. There is deliberately no slot for a digest or a + #: schema directly — a caller managing its own requirements sets ``terms_digest`` on + #: the request, which suppresses the read and says so on the message the signature + #: covers. + registration_requirements: RegistrationRequirementsReader | None = None + #: The RFC 9421 freshness window stamped on every outbound REQUEST signature. + #: + #: Named here as well as on the signer so the knob sits at the same tier it does in + #: Go and TypeScript. A deployment with its own freshness policy sets it; so does one + #: whose peer screens replays on (key id, signature), since timestamps have + #: one-second resolution and two identical requests inside a second otherwise sign to + #: the same bytes — ``monotonic_window`` keeps each signature unique for exactly + #: that, and must be ONE INSTANCE PER CLIENT rather than one per call. + sign_window: Window | None = None #: Mints the X-Request-ID correlation value. ``None`` sends no header. request_id: Callable[[], str] | None = None #: The freshness window stamped on a delivery-fetch proof. @@ -107,6 +148,28 @@ def resolved_verifier(self) -> Verifier: return _NULL_VERIFIER +def _with_requirements_reader( + config: ClientConfig, +) -> tuple[ClientConfig, httpx.Client | None]: + """Fill in the default requirements reader ONCE, and report the transport that + came with it. + + Both client facades call this from their constructor, which is the tier Go resolves + the same default at. Per CALL it would build an SSRF-guarded httpx client for every + registration and close none of them; per CLIENT it is one pooled transport with an + owner that can close it. + + The caller's config is never mutated — a caller may hold it, reuse it across + clients, or read it back — so the filled-in reader rides on a copy. The second + return is the transport this client OWNS: ``None`` when the caller injected a + reader, because then the transport inside it is theirs. + """ + if config.registration_requirements is not None: + return config, None + http = guarded_client() + return replace(config, registration_requirements=WellKnownRequirementsReader(http=http)), http + + class _NullOfferKeyResolver: """Resolves nothing. Under STRICT that rejects every offer, with a reason — the fail-closed default for a client given no key source.""" @@ -472,22 +535,185 @@ def _plan_offer_derived( message: dict[str, Any], idempotency_key: str | None, ) -> Plan: - op = verb.op - sent = _stamp_envelope(op, message, idempotency_key) - # The address is vetted BEFORE the schema: an unroutable recipient is a refusal to - # send, which is a different verdict from a message the server would reject, and the - # caller acts on them differently. - endpoint = vet_exchange_endpoint( - cfg.endpoint_resolver, _str_field(sent, "exchange"), op + # Discovery and execute keep the plain transport, because their address is the + # operator's own configuration. The same split the Go client makes. + sent = _stamp_envelope(verb.op, message, idempotency_key) + return _plan_routed_keyless(cfg, _Routed(verb.op, verb.model, verb.method), sent) + + +# --------------------------------------------------------------------------- +# account setup: register / get account status +# +# They route like a usage report, not like discovery. An account is per-Exchange, +# and which Exchange is the agent's choice PER CALL: a target routinely arrives at +# runtime — a denial names where to register — rather than from configuration. +# +# Neither message carries an idempotency key, so neither verb takes one. +# --------------------------------------------------------------------------- + +#: The ErrorDetail domain for a refusal THIS CLIENT computed, before anything was sent. +#: It names the failing surface, which here is the client's own tier: the Exchange never +#: saw the request, so naming it would attribute a local verdict to a party that reached +#: none. The naming rule the value follows — a Service suffix for an RPC service that +#: exists in the contract, a bare noun for a tier that does not — is recorded on the Go +#: oracle's ``edgeErrorDomain``, beside ``_EDGE_ERROR_DOMAIN``'s twin. +_CLIENT_ERROR_DOMAIN = "ramp.v1.Client" + + +def plan_register(cfg: ClientConfig, request: dict[str, Any]) -> Plan: + """Assemble a registration for the Exchange the request names. + + The caller's identity is the request SIGNATURE. Nothing in the message says who is + registering, and the business payload is not an identity claim. + + Four bounds on ``registration_data`` are checked before anything is signed, in the + order the contract fixes, because a limit that exists to stop work belongs before the + work it would stop — including before the manifest read. + + ``terms_digest`` is filled only when the caller left it ABSENT, from a freshly fetched + manifest, and the payload is pre-checked against the schema that manifest publishes. A + caller that sets the field is managing its own requirements and gets neither. + + A schema this SDK refuses never becomes a local veto: refusing locally and declining + to send would turn a rule about reading a third party's document into a denial of + service against the caller's own user, so an unusable schema is skipped and the + Exchange decides. + """ + op = "register" + sent = _stamp_ver(op, request) + _require_recipient(op, _str_field(sent, "exchange")) + data = sent.get("registration_data") + verdict = check_registration_data(data if isinstance(data, dict) else None) + if verdict != "accepted": + raise malformed(op, f"registration_data: {verdict}") + if sent.get("terms_digest") is None: + _apply_registration_requirements(cfg, op, sent) + return _plan_routed_keyless(cfg, _Routed(op, RegisterRequest, "Register"), sent) + + +def finish_register(plan: Plan, status: int, body: str) -> RegisterResponse: + return decode(plan.op, status, body, RegisterResponse) # type: ignore[no-any-return] + + +def plan_get_account_status(cfg: ClientConfig, request: dict[str, Any]) -> Plan: + """Assemble a status read for the Exchange the request names. + + The request carries no field identifying the caller — the Exchange resolves the + account from the verified signature — so ``exchange`` is the only thing that says + which account is being asked about. + + A caveat worth knowing before calling this in a loop. The request has no varying + field, so two calls to the same Exchange inside one wall-clock second sign IDENTICAL + bytes, and a peer screening replays on (key id, signature) refuses the second. This + verb does not choose the freshness window for you, because a window is one instance + per client rather than per call: set ``ClientConfig.sign_window`` to a + ``monotonic_window`` when repeat calls are expected. + """ + op = "get account status" + sent = _stamp_ver(op, request) + _require_recipient(op, _str_field(sent, "exchange")) + return _plan_routed_keyless( + cfg, _Routed(op, GetAccountStatusRequest, "GetAccountStatus"), sent ) + + +def finish_get_account_status(plan: Plan, status: int, body: str) -> GetAccountStatusResponse: + return decode(plan.op, status, body, GetAccountStatusResponse) # type: ignore[no-any-return] + + +def _apply_registration_requirements(cfg: ClientConfig, op: str, sent: dict[str, Any]) -> None: + """Read what the Exchange asks of a registration and apply it to the request. + + A failed READ refuses the registration rather than sending without a digest. Guessing + here is not the cautious option: an Exchange that publishes a digest refuses a + registration that omits one, so sending anyway trades a local failure the caller can + act on for a remote one it cannot. + """ + # Both client facades fill this in at construction, so the fallback is only for a + # caller driving the plan functions directly. It builds a transport per call, which + # is why the clients do not go through it. + reader = cfg.registration_requirements + if reader is None: + reader = WellKnownRequirementsReader() + exchange = _str_field(sent, "exchange") + try: + reqs = reader.resolve_registration_requirements(exchange) + # Every verdict this seam admits, and each is reachable only through an INJECTED + # reader: the verb's own recipient check runs the host rule first, the SDK's own + # reader raises the middle two itself, and it never raises ManifestUnusableError at + # all. Calling any of them retryable would have a caller retry a verdict. + except ( + ExchangeNotPermittedError, + ManifestNotExchangeError, + ManifestUnusableError, + ) as exc: + # A value this deployment or the Exchange refused is FINAL; anything else is a + # transport failure worth retrying. The same split the routing tier makes, so a + # caller branches on one taxonomy whichever check declined. + raise not_sent(op, str(exc)) from exc + except CallError: + raise + except Exception as exc: # noqa: BLE001 - classified, then re-raised as one shape + if _is_invalid_host_refusal(exc): + raise not_sent(op, str(exc)) from exc + raise CallError(kind=CallErrorKind.UNREACHABLE, op=op, cause=exc) from exc + if reqs.terms_digest is not None: + sent["terms_digest"] = reqs.terms_digest + # A None validator means "nothing to enforce", which is the behaviour the contract + # requires both when the Exchange publishes no schema and when it publishes one this + # SDK refused. One branch, deliberately. + data = sent.get("registration_data") + fails = reqs.schema.validate(data if isinstance(data, dict) else None) if reqs.schema else [] + if fails: + # An empty path addresses the whole object, which is how a missing required + # member and every other whole-object failure is reported. Rendering a bare + # ": ..." there would read as a member with no name. + named = "; ".join( + f"{p}: {f.get('error', '')}" if (p := f.get("path", "")) else f.get("error", "") + for f in fails + ) + # The failures travel as a typed detail, not only as prose. An Exchange attaches + # this same list when it refuses the same payload, so a consumer that renders one + # refusal renders both, and nothing has to parse the members back out of a + # sentence. + raise CallError( + kind=CallErrorKind.MALFORMED, + op=op, + cause=f"registration_data does not match the schema {exchange} publishes: {named}", + detail=registration_failure_detail( + _CLIENT_ERROR_DOMAIN, + "registration_data does not match the published data_schema", + RegistrationFailureReason.REGISTRATION_FAILURE_REASON_INVALID_REGISTRATION_DATA, + fails, + ), + ) + + +@dataclass(frozen=True) +class _Routed: + """One manifest-routed verb that carries no idempotency key: its name, its request + model, and its RPC method.""" + + op: str + model: Any + method: str + + +def _plan_routed_keyless(cfg: ClientConfig, verb: _Routed, sent: dict[str, Any]) -> Plan: + """The routing half every manifest-addressed verb shares, with the envelope already + stamped by the caller. + + It exists so the order — address, then schema, then the guarded leg — is written + once. The address is vetted BEFORE the schema: an unroutable recipient is a refusal to + send, which is a different verdict from a message the server would reject, and the + caller acts on them differently. + """ + op = verb.op + endpoint = vet_exchange_endpoint(cfg.endpoint_resolver, _str_field(sent, "exchange"), op) validate_request(op, sent, verb.model, cfg.validation) - # The offer-derived leg, so the guarded transport: the caller named a domain, the - # manifest it serves named this endpoint, and a signed call now goes there. Discovery - # and execute keep the plain one, because their address is the operator's own - # configuration. The same split the Go client makes. - return _plan( - cfg, _Route(op, endpoint, EXCHANGE_SERVICE, verb.method), sent, guarded=True - ) + # The manifest-derived leg, so the guarded transport: the caller named a domain, the + # manifest it serves named this endpoint, and a signed call now goes there. + return _plan(cfg, _Route(op, endpoint, EXCHANGE_SERVICE, verb.method), sent, guarded=True) # --------------------------------------------------------------------------- @@ -683,7 +909,12 @@ def _plan( op = route.op url = rpc_url(route.base_url, route.service, route.method) body, headers = prepare( - op, url, sent, signer=cfg.signer, request_id=cfg.request_id + op, + url, + sent, + signer=cfg.signer, + request_id=cfg.request_id, + sign_window=cfg.sign_window, ) return Plan( op=op, diff --git a/sdk/python/ramp_sdk/client/content.py b/sdk/python/ramp_sdk/client/content.py index 9417d08b..57d51adb 100644 --- a/sdk/python/ramp_sdk/client/content.py +++ b/sdk/python/ramp_sdk/client/content.py @@ -289,14 +289,45 @@ def edge_refusal(response: httpx.Response, body: bytes) -> CallError: ) +#: The edge's refusal token, mapped to its typed reason. +#: +#: The two vocabularies are separate on purpose — the edge is a code-capable worker with +#: no protobuf runtime, so it answers a string — and ``ramp.proto`` records which token +#: each enum value stands for, beside the value. This table IS that record; it is not +#: derived from the enum's own spelling, and deriving it would be wrong: ``expired`` names +#: URL_EXPIRED, ``pop_expired`` names PROOF_EXPIRED, and neither is the enum suffix +#: lowercased. A port that computed the name instead of reading the record typed two of +#: the eleven tokens and silently dropped the rest. +#: +#: ``missing_sig`` is deliberately ABSENT rather than guessed at: both checkers emit it — +#: the signed-URL check for a missing ``sig`` parameter and the proof check for a missing +#: Signature header — and the enum has a distinct value for each, so the body does not say +#: which ran. An unmapped token still reaches the caller as the raw refusal string; only +#: the typed reason is withheld, which is the honest outcome when the wire cannot say +#: which failure occurred. The edge's parse-level tokens have no enum value at all. +_EDGE_REASON_TOKENS = { + # Signed-URL checks. + "expired": "RETRIEVAL_AUTH_FAILURE_REASON_URL_EXPIRED", + "missing_exp": "RETRIEVAL_AUTH_FAILURE_REASON_URL_EXPIRY_MISSING", + "signature_mismatch": "RETRIEVAL_AUTH_FAILURE_REASON_URL_SIGNATURE_MISMATCH", + # Proof-of-possession checks. + "missing_agent_key": "RETRIEVAL_AUTH_FAILURE_REASON_AGENT_KEY_MISSING", + "keyid_mismatch": "RETRIEVAL_AUTH_FAILURE_REASON_KEYID_MISMATCH", + "thumbprint_mismatch": "RETRIEVAL_AUTH_FAILURE_REASON_THUMBPRINT_MISMATCH", + "pop_missing_created": "RETRIEVAL_AUTH_FAILURE_REASON_PROOF_CREATED_MISSING", + "pop_missing_exp": "RETRIEVAL_AUTH_FAILURE_REASON_PROOF_EXPIRY_MISSING", + "pop_expired": "RETRIEVAL_AUTH_FAILURE_REASON_PROOF_EXPIRED", + "pop_sig_invalid": "RETRIEVAL_AUTH_FAILURE_REASON_PROOF_SIGNATURE_INVALID", +} + + def _typed_reason(token: str) -> Any: - """Promote an edge token to the typed reason when it names one of the protocol's own - values, and answer ``None`` otherwise. The edge's vocabulary and the protocol's are - deliberately separate, and this is the single place they meet.""" - upper = token.upper() - named = upper if upper.startswith("RETRIEVAL_AUTH_FAILURE_REASON_") else ( - f"RETRIEVAL_AUTH_FAILURE_REASON_{upper}" - ) + """Promote an edge token to the typed reason when the protocol records one for it, and + answer ``None`` otherwise. The edge's vocabulary and the protocol's are deliberately + separate, and this is the single place they meet.""" + named = _EDGE_REASON_TOKENS.get(token) + if named is None: + return None try: return retrieval_auth_failure_detail( _EDGE_ERROR_DOMAIN, f"delivery refused: {token}", named # type: ignore[arg-type] diff --git a/sdk/python/ramp_sdk/client/errors.py b/sdk/python/ramp_sdk/client/errors.py index ef37f4f3..ab1e55de 100644 --- a/sdk/python/ramp_sdk/client/errors.py +++ b/sdk/python/ramp_sdk/client/errors.py @@ -67,6 +67,7 @@ def __init__( status: int | None = None, reason: str | None = None, detail: ErrorDetail | None = None, + peer_message: str = "", cause: BaseException | str | None = None, ) -> None: self.kind = kind @@ -78,6 +79,30 @@ def __init__( self.reason = reason #: The typed protocol reason when there is one. self.detail = detail + #: The developer message the peer put on its TYPED reason, when it sent one; + #: ``""`` otherwise. + #: + #: A field rather than something to recover from the rendered string, because a + #: reason rendered into prose cannot be read back out without parsing it. It sits + #: BESIDE ``reason`` rather than in it: ``reason`` is the peer's machine token. + #: + #: Deliberately NOT filled from the transport envelope when there is no typed + #: detail. An answer that did not come from a RAMP service carries no message of + #: its own, and the text a transport synthesizes for one is that transport's — + #: connect-go writes a status line where this client writes nothing — so carrying + #: it would make the value a property of the language rather than of the answer. + #: That text stays reachable through ``cause``. + #: + #: NON-AUTHORITATIVE and UNBOUNDED — the contract says both of the field it comes + #: from. Branch on ``kind`` or on the typed reason, never on this text, and bound + #: it before rendering it to a log line or an agent. + #: + #: Taken ONLY from what the decode site passed, never derived from ``detail`` + #: here. A detail is not always the peer's: the content leg SYNTHESIZES one from + #: the edge's refusal token, and its message is this SDK's own sentence with the + #: token quoted into it. Deriving from the field would put the SDK's words in the + #: field that claims to hold a remote party's. + self.peer_message = peer_message self.cause = cause super().__init__(_render(kind, op, status, reason, cause)) diff --git a/sdk/python/ramp_sdk/client/route.py b/sdk/python/ramp_sdk/client/route.py index 1d991f31..4cb7a74c 100644 --- a/sdk/python/ramp_sdk/client/route.py +++ b/sdk/python/ramp_sdk/client/route.py @@ -13,9 +13,10 @@ from __future__ import annotations -from typing import Protocol, runtime_checkable +from typing import Any, Protocol, runtime_checkable from ramp_sdk._endpoint_rule import endpoint_refusal +from ramp_sdk._hostref import _is_invalid_host_refusal from ramp_sdk._hostref import _redact_userinfo as redact_userinfo from ramp_sdk.hosts import is_bare_host from ramp_sdk.resolvers.errors import ( @@ -41,9 +42,10 @@ class EndpointResolver(Protocol): not accept, advertises no endpoint, or advertises one that must not be used — MUST raise :class:`~ramp_sdk.resolvers.errors.ManifestVersionRefusedError`, :class:`~ramp_sdk.resolvers.errors.NoEndpointError`, - :class:`~ramp_sdk.resolvers.errors.EndpointRefusedError`, or the ``ValueError`` - ``is_bare_host`` raises; those surface as ``NOT_SENT``, which tells the caller not to - retry. Anything else is read as a transport failure and reported as ``UNREACHABLE``, + :class:`~ramp_sdk.resolvers.errors.EndpointRefusedError`, or the invalid-host + ``ValueError`` ``is_bare_host`` raises — recognised by its wording, so an unrelated + ``ValueError`` is NOT one; those surface as ``NOT_SENT``, which tells the caller not + to retry. Anything else is read as a transport failure and reported as ``UNREACHABLE``, i.e. worth retrying. An implementation that raises a bare exception for a refusal therefore has its final answer retried indefinitely. """ @@ -97,16 +99,19 @@ def vet_exchange_endpoint( # allowed, the manifest was read and advertises no endpoint at all, or it # advertises one the resolver will not hand back. # - # ValueError is in the set because the resolver checks the host itself too, and a - # value that is not a host will not become one on a later attempt. This module - # checks it before resolving, so the SDK's own resolver never reaches here that - # way — an injected one can. + # The invalid-host refusal is in the set because the resolver checks the host + # itself too, and a value that is not a host will not become one on a later + # attempt. This module checks it before resolving, so the SDK's own resolver + # never reaches here that way — an injected one can. It is recognised through + # the one predicate the account leg also uses: catching the bare ValueError it + # is raised as would call every other ValueError final too, and a document a + # reader could not parse is a transport failure in the oracle and in TypeScript. kind = ( CallErrorKind.NOT_SENT if isinstance( - exc, - ManifestVersionRefusedError | NoEndpointError | EndpointRefusedError | ValueError, + exc, ManifestVersionRefusedError | NoEndpointError | EndpointRefusedError ) + or _is_invalid_host_refusal(exc) else CallErrorKind.UNREACHABLE ) raise CallError( @@ -130,3 +135,39 @@ def vet_exchange_endpoint( f"{redact_userinfo(exchange_domain)!r} advertises: {refusal}", ) return endpoint + + +class RegistrationRequirementsReader(Protocol): + """Reports what one Exchange asks of a registration. + + A Protocol for the same two reasons the endpoint seam is one: a test can drive a + registration without standing up a manifest server, and this module has no way to + accept a terms digest or a schema from configuration — the only way to skip the read + is to set ``terms_digest`` on the request, where the signature covers it. + + An implementation MUST NOT serve the answer from a cache. The contract requires a + registering client to read the digest from a freshly fetched manifest, so a cached + one breaks the rule the field exists to record. + + An implementation's FAILURE decides how a caller is told to react, so it is part of + the contract rather than an implementation detail. A failure that is a VERDICT — the + domain is unusable, the deployment excludes it, the document served is not an + Exchange's, or it is one this reader cannot use — MUST raise + :class:`~ramp_sdk.resolvers.errors.ExchangeNotPermittedError`, + :class:`~ramp_sdk.resolvers.errors.ManifestNotExchangeError`, + :class:`~ramp_sdk.resolvers.errors.ManifestUnusableError`, or the invalid-host + ``ValueError`` raised for a value that is not a bare domain — recognised by its + wording, so an unrelated ``ValueError`` is NOT one; those surface as ``NOT_SENT``, + which tells the caller not to retry. Anything else is read as a transport failure + and reported as ``UNREACHABLE``, i.e. worth retrying. An implementation that raises + a bare exception for a refusal therefore has its final answer retried indefinitely. + + ``ManifestUnusableError`` is the one an implementation stricter than the SDK's own + reaches for. That reader refuses three things and treats every other disappointment + as absence or as a transport failure; one that validates the whole document, or + refuses a version, is holding a final answer this seam would otherwise report as + transient. + """ + + def resolve_registration_requirements(self, exchange: str) -> Any: # pragma: no cover + ... diff --git a/sdk/python/ramp_sdk/resolvers/__init__.py b/sdk/python/ramp_sdk/resolvers/__init__.py index 72d26460..495449c7 100644 --- a/sdk/python/ramp_sdk/resolvers/__init__.py +++ b/sdk/python/ramp_sdk/resolvers/__init__.py @@ -21,8 +21,11 @@ from ramp_sdk.resolvers.errors import ( DirectoryUnavailableError, EndpointRefusedError, + ExchangeNotPermittedError, KeyExpiredError, KeyRevokedError, + ManifestNotExchangeError, + ManifestUnusableError, ManifestVersionRefusedError, NoEndpointError, ResolverError, @@ -30,6 +33,10 @@ UnknownKeyError, ) from ramp_sdk.resolvers.offer_key_cache import CachedOfferKeyResolver, DirectoryFetch +from ramp_sdk.resolvers.registration_requirements import ( + RegistrationRequirements, + WellKnownRequirementsReader, +) from ramp_sdk.resolvers.wba import ( WBA_DIRECTORY_PATH, WBAKeyResolver, @@ -47,10 +54,14 @@ "DirectoryFetch", "DirectoryUnavailableError", "EndpointRefusedError", + "ExchangeNotPermittedError", "KeyExpiredError", "KeyRevokedError", + "ManifestNotExchangeError", + "ManifestUnusableError", "ManifestVersionRefusedError", "NoEndpointError", + "RegistrationRequirements", "ResolverError", "RevocationUnevaluatedError", "SsrfError", @@ -58,6 +69,7 @@ "WBAKeyResolver", "WellKnownEndpointResolver", "WellKnownKeyResolver", + "WellKnownRequirementsReader", "active_ed25519_key", "active_ed25519_key_screened", "active_ed25519_key_with_expiry", diff --git a/sdk/python/ramp_sdk/resolvers/_http.py b/sdk/python/ramp_sdk/resolvers/_http.py index fc380204..60da0acb 100644 --- a/sdk/python/ramp_sdk/resolvers/_http.py +++ b/sdk/python/ramp_sdk/resolvers/_http.py @@ -297,11 +297,20 @@ def guarded_async_client(**httpx_kwargs: Any) -> httpx.AsyncClient: def default_client() -> httpx.Client: - """The wellknown resolvers' default: a plain (UNGUARDED) client. - - Their URL is a static caller-configured value (not request-derived), so an - SSRF guard is not warranted — matching the Go oracle, which guards only the - WBA client. + """The default for a resolver whose URL is FIXED and operator-chosen: a plain + (UNGUARDED) client. + + Which default a resolver takes follows its URL's PROVENANCE, and that is the whole + rule — the Go oracle states it in the options struct these ports mirror. A fixed + operator-chosen address (the well-known JWKS) takes this client; an on-prem JWKS + may legitimately be private, and the operator rather than an attacker chose it. A + REQUEST-DERIVED host — the WBA directory named by a Signature-Agent header, an + Exchange domain read off an offer or a registration — takes ``guarded_client``, + because the party choosing the address is not the party running the process. + + The endpoint resolver in this package is request-derived and still defaults here, + which is a known gap being closed separately; it is not the rule. Do not reach for + this client for a new resolver without first asking where its URL comes from. """ return httpx.Client( follow_redirects=True, max_redirects=_ssrf.MAX_REDIRECTS, timeout=_TIMEOUT_S diff --git a/sdk/python/ramp_sdk/resolvers/errors.py b/sdk/python/ramp_sdk/resolvers/errors.py index 8d6a00bf..be495d77 100644 --- a/sdk/python/ramp_sdk/resolvers/errors.py +++ b/sdk/python/ramp_sdk/resolvers/errors.py @@ -66,6 +66,45 @@ class EndpointRefusedError(ResolverError): """ +class ExchangeNotPermittedError(ResolverError): + """The deployment's allow overlay excluded this Exchange domain, before anything + was dialled. + + It says nothing about whether the Exchange exists or answers, only that this + deployment declined to ask, so the remedy is a configuration change rather than a + retry. Peer of Go ``ErrExchangeNotPermitted`` / TS ``ExchangeNotPermitted``. + """ + + +class ManifestNotExchangeError(ResolverError): + """The document served at the domain's well-known path describes some other role. + + Registration requirements are an Exchange's to publish, so a manifest claiming to + be an agent, a broker or a publisher is refused rather than read for members it + has no business carrying. A manifest naming no role at all is refused the same + way: the field is required by the contract, and reading silence as assent would + make the check advisory. Peer of Go ``ErrManifestNotExchange`` / TS + ``ManifestNotExchange``. + """ + + +class ManifestUnusableError(ResolverError): + """The document arrived and this reader cannot use it. + + A VERDICT and not a failed read: the bytes were served, and the next attempt gets + the same ones, so a caller told to retry retries forever. + + The SDK's own :class:`~ramp_sdk.resolvers.WellKnownRequirementsReader` never raises + it. The two ways a manifest can disappoint that reader are both deliberate + non-errors: a member carrying a type the contract does not admit reads as ABSENT, + because the projection is shared with the endpoint and key faces, and a document + that does not decode at all is a transport failure. This class exists because the + reader seam is injectable and one stricter than the SDK's own — validating the whole + document, or refusing a version — has to be able to say its refusal is final. Peer + of Go ``ErrManifestUnusable`` / TS ``ManifestUnusable``. + """ + + class ManifestVersionRefusedError(ResolverError): """A ``/.well-known/ramp.json`` was fetched and parsed but carries a ``WellKnownManifest.ver`` this resolver does not accept: an unrecognised major diff --git a/sdk/python/ramp_sdk/resolvers/registration_requirements.py b/sdk/python/ramp_sdk/resolvers/registration_requirements.py new file mode 100644 index 00000000..25d66c49 --- /dev/null +++ b/sdk/python/ramp_sdk/resolvers/registration_requirements.py @@ -0,0 +1,237 @@ +"""Reading what one Exchange asks of a registration — the terms revision that +submitting one accepts, and the schema its registration_data must match — out of that +Exchange's own ``/.well-known/ramp.json``. Python port of the Go oracle +(``resolvers/registrationrequirements.go``). + +EVERY READ IS A FRESH FETCH, and that is the whole design. The protocol requires it in +as many words: a registering client MUST read the terms digest from a freshly fetched +manifest rather than a cached copy. A cached ENDPOINT is fine — a wrong one fails +loudly — but a cached DIGEST is not, because a client cannot detect staleness locally, +so a warm cache would make it echo a value the Exchange has already stopped accepting +and retry the same refusal until the cache expired. + +That is why this is a SEPARATE face rather than a third method on the endpoint +resolver. That resolver is built out of exactly the mechanism this value may not touch +— a per-host TTL cache with single-flight coalescing on top — and it exposes no bypass. +A face that holds no document cache leaves no cache slot to reuse, which is what makes +the rule structural rather than a convention callers remember. + +Not even the compiled validator is held. Memoising it is a cache, and the useful key +for one is a property of a deployment's threat model rather than of the protocol; an +application that wants it wraps this face. +""" + +from __future__ import annotations + +import json +from dataclasses import dataclass +from typing import Any + +import httpx + +from ramp_sdk.hosts import is_bare_domain +from ramp_sdk.regschema import ( + RegistrationSchema, + SchemaVerdict, + compile_registration_schema, +) +from ramp_sdk.resolvers._http import fetch_strict, guarded_client +from ramp_sdk.resolvers.errors import ( + DirectoryUnavailableError, + ExchangeNotPermittedError, + ManifestNotExchangeError, +) +from ramp_sdk._hostref import _invalid_host +from ramp_sdk.resolvers.wellknown import AllowFn + +__all__ = [ + "RegistrationRequirements", + "WellKnownRequirementsReader", +] + + +@dataclass(frozen=True) +class RegistrationRequirements: + """What one Exchange asks of a registration. + + Both members are optional in the contract, and their absence is a normal answer + rather than a failure: an Exchange publishing neither accepts registration data + uninspected and records no terms acceptance. + """ + + #: The manifest's ``terms_digest``, ``None`` when the Exchange publishes none. Copy + #: it onto ``RegisterRequest.terms_digest`` unchanged: the request signature covers + #: the echo, and that echo is the durable record of which terms revision the + #: operator accepted. + terms_digest: str | None = None + #: Validates ``registration_data`` before anything is signed. ``None`` in TWO cases + #: — the Exchange publishes none, and it publishes one this SDK refuses — and + #: ``verdict`` is what tells them apart. Both are deliberately the same VALUE, + #: because the contract requires a client that cannot check locally to send anyway: + #: a local check that cannot run must not become a local veto. + schema: RegistrationSchema | None = None + #: The SDK's answer for the published schema. ``NOT_PUBLISHED`` is the ordinary + #: absent case; ``ACCEPTED`` means ``schema`` is usable; anything else names why a + #: published schema was refused, which is worth logging and is never worth refusing + #: the registration over. + verdict: SchemaVerdict = "not_published" + + +class WellKnownRequirementsReader: + """Read registration requirements from an Exchange's own well-known manifest. + + ``ttl`` and ``now`` are deliberately absent from the constructor: this face caches + nothing, so it has no freshness to compute. + """ + + def __init__( + self, + *, + scheme: str = "https", + http: httpx.Client | None = None, + allow: AllowFn | None = None, + ) -> None: + self._scheme = scheme or "https" + # The domain is CALLER-NAMED — an agent registers at whichever Exchange it + # means to transact with, and that domain routinely arrives at runtime rather + # than from configuration. That is REQUEST-DERIVED provenance, which takes the + # SSRF-guarded transport; a fixed operator-chosen URL is what takes the plain + # one. A deployment that must reach a private or loopback Exchange injects its + # own client here, or opts out through the SKIP_SSRF / ALLOW_INSECURE flags. + self._http = http if http is not None else guarded_client() + self._allow = allow + + def resolve_registration_requirements(self, exchange: str) -> RegistrationRequirements: + """Fetch ``exchange``'s manifest and report what it asks of a registration. + + The answer is never served from a cache; see the module doc for why that is the + point rather than an omission. + + Two refusals come before anything is dialled, and both are here rather than at + the call site because they are properties of building this URL and of this + deployment's policy, not of any one caller's plans. + + The SHAPE predicate is the contract's ``is_bare_domain``, not the routing tier's + ``is_bare_host``. The two are kept apart deliberately and this leg is where the + difference bites: nothing upstream has run the contract's rule, and the URL + below is built by concatenation, so a value carrying a path or userinfo would + choose WHAT is fetched rather than merely where from. + + A refused schema is never an exception: the verdict is returned alongside a + ``None`` schema, because the contract requires a client that cannot check + locally to send anyway and let the Exchange decide. + """ + if not is_bare_domain(exchange): + raise _invalid_host(exchange, "not a bare domain") + if self._allow is not None and not self._allow(exchange): + raise ExchangeNotPermittedError(f"exchange {exchange} not permitted by policy") + url = f"{self._scheme}://{exchange}/.well-known/ramp.json" + raw_body = fetch_strict(self._http, url) + try: + # Decoded ONCE, and the member below is sliced out of this text and + # re-encoded. The round trip is exact for valid UTF-8 — including a byte + # order mark, which the schema rules refuse and which must therefore + # survive to reach them — and invalid UTF-8 raises here, where it reads as + # an undecodable manifest rather than as a schema fault. + body = raw_body.decode("utf-8") + doc = json.loads(body) + except ValueError as exc: + raise DirectoryUnavailableError(f"manifest decode {url}") from exc + if not isinstance(doc, dict): + raise DirectoryUnavailableError(f"manifest at {url} is not a JSON object") + if not _describes_exchange(doc): + raise ManifestNotExchangeError(f"host={exchange}") + digest = doc.get("terms_digest") + # Compiled from the bytes AS SERVED, because every cap the registration-schema + # rules state is defined over those bytes. The member is sliced out of the body + # rather than re-serialised from the parsed value: json.loads discards + # whitespace and reformats numbers, and a document that minifies under the cap + # while exceeding it on the wire would be refused by an Exchange and accepted + # here — the two-privately-chosen-limits failure the contract's numbers exist to + # prevent. + raw = _raw_member(body, "account_registration", "data_schema") + schema: RegistrationSchema | None = None + verdict: SchemaVerdict = "not_published" + if raw is not None: + schema, verdict = compile_registration_schema(raw) + return RegistrationRequirements( + terms_digest=digest if isinstance(digest, str) else None, + schema=schema, + verdict=verdict, + ) + + +def _describes_exchange(doc: dict[str, Any]) -> bool: + """proto-JSON renders an enum as its name or its number, so both are accepted. An + absent or unrecognised role is not an Exchange.""" + role = doc.get("role") + return role == "ROLE_EXCHANGE" or role == 2 # noqa: PLR1714 - two distinct wire forms + + +_DECODER = json.JSONDecoder() +#: RFC 8259 whitespace, and nothing wider — the same four bytes the blank-schema rule +#: counts. +_WS = " \t\r\n" + + +def _raw_member(body: str, *path: str) -> bytes | None: + """Return the EXACT served bytes of a nested object member, or ``None`` when any + step of the path is absent. + + ``body`` has already parsed as JSON when this runs, so the walk below is over + known-well-formed input and needs no error path — it returns ``None`` for anything + it cannot walk, and the caller reads that as "no schema published", which is the + safe direction. + + Value extents come from ``JSONDecoder.raw_decode`` rather than a hand-rolled + skipper: it already knows where every JSON value ends, and a second implementation + of that is a second place to get strings and escapes wrong. + """ + start = 0 + for step, key in enumerate(path): + found = _member_extent(body, start, key) + if found is None: + return None + if step == len(path) - 1: + return body[found[0] : found[1]].encode("utf-8") + start = found[0] + return None + + +def _member_extent(body: str, start: int, key: str) -> tuple[int, int] | None: + """Find ``key`` in the object beginning at ``start`` and return the ``[start, end)`` + of its VALUE.""" + i = _skip_ws(body, start) + if i >= len(body) or body[i] != "{": + return None + i = _skip_ws(body, i + 1) + if i < len(body) and body[i] == "}": + return None + while True: + try: + name, i = _DECODER.raw_decode(body, i) + except ValueError: + return None + if not isinstance(name, str): + return None + i = _skip_ws(body, i) + if i >= len(body) or body[i] != ":": + return None + value_start = _skip_ws(body, i + 1) + try: + _, value_end = _DECODER.raw_decode(body, value_start) + except ValueError: + return None + if name == key: + return value_start, value_end + i = _skip_ws(body, value_end) + if i < len(body) and body[i] == ",": + i = _skip_ws(body, i + 1) + continue + return None + + +def _skip_ws(body: str, i: int) -> int: + while i < len(body) and body[i] in _WS: + i += 1 + return i diff --git a/sdk/python/ramp_sdk/signing_transport.py b/sdk/python/ramp_sdk/signing_transport.py index e4093047..c947b77a 100644 --- a/sdk/python/ramp_sdk/signing_transport.py +++ b/sdk/python/ramp_sdk/signing_transport.py @@ -153,6 +153,7 @@ def sign_outbound( url: str, body: bytes, authorization: str, + window: Window | None = None, ) -> SignedOutbound: """Sign an outbound request; return it with the RFC 9421 headers attached. @@ -160,8 +161,14 @@ def sign_outbound( token, which BINDS that emptiness (mirrors the L1 sign_request contract). The header is still emitted and still sent; an absent header is a different thing entirely and a verifier refuses it. + + ``window`` overrides the one this transport was built with, so a client can + surface the freshness knob at the tier Go and TypeScript surface it at without + the application having to rebuild its signer. It must be ONE INSTANCE held by the + caller rather than a fresh one per call: a monotonic window carries the running + maximum that makes each signature unique, and one built per request has none. """ - created, expires = self._window() + created, expires = (window or self._window)() signed = sign_request( method=method, url=url, diff --git a/sdk/python/ramp_sdk/sync/__init__.py b/sdk/python/ramp_sdk/sync/__init__.py index c8422ec9..59b41cfa 100644 --- a/sdk/python/ramp_sdk/sync/__init__.py +++ b/sdk/python/ramp_sdk/sync/__init__.py @@ -26,7 +26,7 @@ require_dialable_scheme, rpc_headers, ) -from ramp_sdk.client._verbs import ClientConfig +from ramp_sdk.client._verbs import ClientConfig, _with_requirements_reader from ramp_sdk.client.content import ( MAX_ERROR_BODY_BYTES, Content, @@ -41,8 +41,10 @@ if TYPE_CHECKING: from wire.models import ( DisputeResponse, + GetAccountStatusResponse, PushResourcesResponse, RefreshCatalogResponse, + RegisterResponse, RemoveResourcesResponse, TransactionResponse, UsageReportResponse, @@ -67,7 +69,6 @@ def __init__( http: httpx.Client | None, guarded: httpx.Client | None = None, ) -> None: - self._config = config self._owns = http is None self._http = http if http is not None else httpx.Client( follow_redirects=False, trust_env=False @@ -78,9 +79,15 @@ def __init__( self._guarded = guarded if guarded is not None else ( self._http if not self._owns else guarded_client(follow_redirects=False) ) + self._config, self._requirements_http = _with_requirements_reader(config) def close(self) -> None: """Close the transports this client built. An injected one is left alone.""" + # Independent of the RPC legs above: this client is built here whenever the + # caller injected no reader, whether or not it injected an RPC transport, so it + # is closed on its own terms rather than behind that ownership question. + if self._requirements_http is not None: + self._requirements_http.close() if not self._owns: return self._http.close() @@ -171,6 +178,16 @@ def dispute( status, body = self._send(plan) return _verbs.finish_dispute(plan, status, body) + def register(self, request: dict[str, Any]) -> RegisterResponse: + plan = _verbs.plan_register(self._config, request) + status, body = self._send(plan) + return _verbs.finish_register(plan, status, body) + + def get_account_status(self, request: dict[str, Any]) -> GetAccountStatusResponse: + plan = _verbs.plan_get_account_status(self._config, request) + status, body = self._send(plan) + return _verbs.finish_get_account_status(plan, status, body) + def fetch(self, signed_url: str) -> Content: headers, timeout, max_bytes = _fetch_inputs(self._config, signed_url) op = "fetch content" diff --git a/sdk/python/ramp_sdk/window.py b/sdk/python/ramp_sdk/window.py index 5acb471b..0274284d 100644 --- a/sdk/python/ramp_sdk/window.py +++ b/sdk/python/ramp_sdk/window.py @@ -42,6 +42,11 @@ def monotonic_window(now: Callable[[], float], ttl_sec: int) -> Window: relay requests from colliding in the server's replay store. ``created`` tracks ``int(now())``, so the pair stays clock-consistent. Thread-safe: the running maximum is guarded by a lock. + + ONE INSTANCE PER CLIENT, never one per call. The running maximum is the whole + mechanism: a window built per request starts from zero, cannot see the + previous signature, and provides exactly none of the uniqueness it was chosen + for — while still looking correct at the call site. """ lock = threading.Lock() last_expires = 0 diff --git a/sdk/python/tests/test_account.py b/sdk/python/tests/test_account.py new file mode 100644 index 00000000..aa5c52ea --- /dev/null +++ b/sdk/python/tests/test_account.py @@ -0,0 +1,483 @@ +"""The account-setup verbs, on both faces. + +They route like a usage report, not like discovery: the destination is read off the +request's own ``exchange`` field and resolved through that Exchange's own manifest. +""" + +from __future__ import annotations + +import asyncio +import json +from typing import Any + +import httpx +import pytest +from test_client import FACES, _IDS, Face, Recorder, _config # noqa: PLC2701 + +from ramp_sdk._hostref import _invalid_host # noqa: PLC2701 +from ramp_sdk.client import CallError, CallErrorKind +from ramp_sdk.regschema import compile_registration_schema +from ramp_sdk.resolvers import ( + ExchangeNotPermittedError, + ManifestNotExchangeError, + ManifestUnusableError, + RegistrationRequirements, + WellKnownRequirementsReader, +) + +_ENDPOINT = "https://exchange.test" +_DIGEST = "sha256:" + "ab" * 32 +_SCHEMA = ( + b'{"type":"object","required":["legal_entity"],' + b'"properties":{"legal_entity":{"type":"string"}}}' +) +#: Fails two ways at once: a required member is absent, which is a whole-object violation +#: the wire reports with an EMPTY path, and a member that IS present breaks its pattern, +#: which is reported against its own pointer. One refusal carrying both is what a consumer +#: has to render, and the empty path is the half easiest to render wrong. +_BOTH_SHAPES_SCHEMA = ( + b'{"type":"object","required":["legal_entity"],' + b'"properties":{"vat_id":{"type":"string","pattern":"^[A-Z]{2}[0-9]+$"}}}' +) + + +class _Resolver: + def resolve_endpoint(self, _host: str) -> str: + return _ENDPOINT + + +class _Requirements: + """A reader that answers a fixed set of requirements, or raises.""" + + def __init__(self, reqs: RegistrationRequirements | Exception) -> None: + self._reqs = reqs + self.calls = 0 + + def resolve_registration_requirements(self, _exchange: str) -> RegistrationRequirements: + self.calls += 1 + if isinstance(self._reqs, Exception): + raise self._reqs + return self._reqs + + +def _account_config(reqs: Any = None, **overrides: Any) -> Any: + return _config( + endpoint_resolver=_Resolver(), + registration_requirements=reqs if reqs is not None else _Requirements( + RegistrationRequirements() + ), + **overrides, + ) + + +def _refused_by_the_schema(face: Face, schema_source: bytes, payload: dict[str, Any]) -> CallError: + """Drive a pre-check refusal against ``schema_source``, so the tests below share one + arrangement and differ only in what they read off the failure.""" + schema, verdict = compile_registration_schema(schema_source) + assert verdict == "accepted" + reqs = _Requirements(RegistrationRequirements(schema=schema, verdict=verdict)) + rec = Recorder({"ver": "1.0", "billing_ref": "acct-1"}) + + with pytest.raises(CallError) as caught: + face.run( + face.client(_account_config(reqs), rec).register( + {"exchange": "exchange.test", "registration_data": payload} + ) + ) + assert caught.value.kind is CallErrorKind.MALFORMED + assert rec.seen == [] + return caught.value + + +@pytest.mark.parametrize("face", FACES, ids=_IDS) +def test_register_addresses_the_rpc_and_stamps_ver(face: Face) -> None: + rec = Recorder({"ver": "1.0", "billing_ref": "acct-1", "active": True}) + client = face.client(_account_config(), rec) + resp = face.run( + client.register( + {"exchange": "exchange.test", "registration_data": {"legal_entity": "Acme"}} + ) + ) + assert resp.billing_ref == "acct-1" + assert rec.seen[0].url.path == "/ramp.v1.ExchangeService/Register" + body = rec.body() + assert body["ver"] == "1.0" + assert body["exchange"] == "exchange.test" + # No idempotency key is minted: the message carries none. + assert "idempotency_key" not in body + + +@pytest.mark.parametrize("face", FACES, ids=_IDS) +def test_register_echoes_the_freshly_fetched_digest(face: Face) -> None: + rec = Recorder({"ver": "1.0", "billing_ref": "acct-1"}) + reqs = _Requirements(RegistrationRequirements(terms_digest=_DIGEST)) + request: dict[str, Any] = {"exchange": "exchange.test"} + face.run(face.client(_account_config(reqs), rec).register(request)) + assert rec.body()["terms_digest"] == _DIGEST + assert reqs.calls == 1 + # The caller's message crossed a boundary as an argument, not as a buffer to fill. + assert "terms_digest" not in request + + +@pytest.mark.parametrize("face", FACES, ids=_IDS) +def test_register_a_caller_supplied_digest_suppresses_the_read(face: Face) -> None: + rec = Recorder({"ver": "1.0", "billing_ref": "acct-1"}) + # The reader raises if it is consulted at all, which is how this asserts the read + # does not happen. + reqs = _Requirements(AssertionError("the requirements read must not happen")) + mine = "sha256:" + "cd" * 32 + face.run( + face.client(_account_config(reqs), rec).register( + {"exchange": "exchange.test", "terms_digest": mine} + ) + ) + assert rec.body()["terms_digest"] == mine + assert reqs.calls == 0 + + +@pytest.mark.parametrize("face", FACES, ids=_IDS) +def test_register_pre_checks_the_published_schema(face: Face) -> None: + schema, verdict = compile_registration_schema(_SCHEMA) + assert verdict == "accepted" + reqs = _Requirements(RegistrationRequirements(schema=schema, verdict=verdict)) + rec = Recorder({"ver": "1.0", "billing_ref": "acct-1"}) + + with pytest.raises(CallError) as caught: + face.run( + face.client(_account_config(reqs), rec).register( + {"exchange": "exchange.test", "registration_data": {"trading_name": "Acme"}} + ) + ) + assert caught.value.kind is CallErrorKind.MALFORMED + assert "legal_entity" in str(caught.value) + assert rec.seen == [] + + # The conforming payload goes through against the same schema. + face.run( + face.client(_account_config(reqs), rec).register( + {"exchange": "exchange.test", "registration_data": {"legal_entity": "Acme"}} + ) + ) + assert len(rec.seen) == 1 + + +@pytest.mark.parametrize("face", FACES, ids=_IDS) +def test_register_pre_check_refusal_carries_its_field_errors(face: Face) -> None: + """What the pre-check computed is what the caller gets. An Exchange attaches this same + list, from this same validator, when it refuses the same payload — so a consumer that + renders one refusal renders both, and nothing has to parse the members back out of a + sentence.""" + err = _refused_by_the_schema(face, _BOTH_SHAPES_SCHEMA, {"vat_id": "de1"}) + + assert err.detail is not None + failure = err.detail.registration_failure + assert failure is not None + assert failure.reason.value == "REGISTRATION_FAILURE_REASON_INVALID_REGISTRATION_DATA" + # The PATHS are what this asserts, never the constraint prose beside them: that text + # comes from each language's validator library, and the contract calls it + # non-authoritative for exactly that reason. + assert [f.path or "" for f in failure.field_errors or []] == ["", "/vat_id"] + + +@pytest.mark.parametrize("face", FACES, ids=_IDS) +def test_register_renders_a_whole_object_violation_with_no_member_name(face: Face) -> None: + """An empty path addresses the whole object, which is how a missing required member is + reported. Rendering a bare ``": ..."`` there would read as a member with no name.""" + err = _refused_by_the_schema(face, _BOTH_SHAPES_SCHEMA, {"vat_id": "de1"}) + + assert "publishes: : " not in str(err) + # The member that HAS a name still carries it, so the fix did not drop the separator + # everywhere. + assert "/vat_id: " in str(err) + + +@pytest.mark.parametrize("face", FACES, ids=_IDS) +def test_register_an_unusable_schema_does_not_block_the_send(face: Face) -> None: + """Refusing locally would turn a rule about reading a third party's document into a + denial of service against the caller's own user.""" + schema, verdict = compile_registration_schema( + b'{"$schema":"https://json-schema.org/draft/2019-09/schema","type":"object"}' + ) + assert schema is None + assert verdict == "wrong_dialect" + rec = Recorder({"ver": "1.0", "billing_ref": "acct-1"}) + reqs = _Requirements(RegistrationRequirements(schema=schema, verdict=verdict)) + face.run( + face.client(_account_config(reqs), rec).register( + {"exchange": "exchange.test", "registration_data": {"anything": "goes"}} + ) + ) + assert len(rec.seen) == 1 + + +@pytest.mark.parametrize("face", FACES, ids=_IDS) +def test_register_refuses_an_out_of_bounds_payload_before_the_read(face: Face) -> None: + """A limit that exists to stop work runs before the work it would stop.""" + rec = Recorder({}) + reqs = _Requirements(AssertionError("the bound must run before the read")) + too_many = {f"m{i}": "v" for i in range(65)} + with pytest.raises(CallError) as caught: + face.run( + face.client(_account_config(reqs), rec).register( + {"exchange": "exchange.test", "registration_data": too_many} + ) + ) + assert caught.value.kind is CallErrorKind.MALFORMED + assert rec.seen == [] + assert reqs.calls == 0 + + +@pytest.mark.parametrize("face", FACES, ids=_IDS) +def test_register_classifies_a_refused_requirements_read_as_final(face: Face) -> None: + rec = Recorder({}) + for raised, kind in ( + (ExchangeNotPermittedError("blocked"), CallErrorKind.NOT_SENT), + (ManifestNotExchangeError("wrong role"), CallErrorKind.NOT_SENT), + # The SDK's own reader never raises this one: it reads an off-spec member as + # absent and treats an undecodable document as a transport failure. A stricter + # INJECTED reader raises it, and the document arrives unusable again next time. + (ManifestUnusableError('ver "2.0" has major 2'), CallErrorKind.NOT_SENT), + # Reachable only through an INJECTED reader with its own host rule — the verb's + # own recipient check runs this one first. Retryable would be wrong: a value that + # is not a host will not become one on a later attempt. + (_invalid_host("exchange.test/path", "not a bare domain"), CallErrorKind.NOT_SENT), + (RuntimeError("connection reset"), CallErrorKind.UNREACHABLE), + # A ValueError that is NOT the invalid-host refusal. json.JSONDecodeError + # subclasses ValueError, so catching the bare type would call a document an + # injected reader could not parse final — where the oracle and TypeScript both + # call it a transport failure. + (ValueError("expecting value: line 1 column 1"), CallErrorKind.UNREACHABLE), + ): + with pytest.raises(CallError) as caught: + face.run( + face.client(_account_config(_Requirements(raised)), rec).register( + {"exchange": "exchange.test"} + ) + ) + assert caught.value.kind is kind, raised + assert rec.seen == [] + + +@pytest.mark.parametrize("face", FACES, ids=_IDS) +def test_get_account_status_reads_an_accountless_answer_as_normal(face: Face) -> None: + rec = Recorder({"ver": "1.0", "billing_ref": "", "active": False}) + resp = face.run( + face.client(_account_config(), rec).get_account_status({"exchange": "exchange.test"}) + ) + assert resp.billing_ref == "" + assert resp.active is False + assert rec.seen[0].url.path == "/ramp.v1.ExchangeService/GetAccountStatus" + assert rec.body()["ver"] == "1.0" + + +@pytest.mark.parametrize("face", FACES, ids=_IDS) +def test_account_verbs_refuse_an_unaddressed_request(face: Face) -> None: + rec = Recorder({}) + client = face.client(_account_config(), rec) + for exchange in ("", "https://exchange.test", "exchange.test/path", "ex_change.test"): + for call in ( + lambda e=exchange: client.register({"exchange": e}), + lambda e=exchange: client.get_account_status({"exchange": e}), + ): + with pytest.raises(CallError) as caught: + face.run(call()) + assert caught.value.kind is CallErrorKind.NOT_SENT, exchange + assert caught.value.status is None + assert rec.seen == [] + + +@pytest.mark.parametrize("face", FACES, ids=_IDS) +def test_account_verbs_caller_ver_wins(face: Face) -> None: + rec = Recorder({"ver": "1.0", "billing_ref": "acct-1"}) + client = face.client(_account_config(), rec) + face.run(client.register({"exchange": "exchange.test", "ver": "9.9"})) + assert rec.body(0)["ver"] == "9.9" + face.run(client.get_account_status({"exchange": "exchange.test", "ver": "9.9"})) + assert rec.body(1)["ver"] == "9.9" + + +def test_peer_message_stays_empty_for_a_detail_the_sdk_synthesized() -> None: + """The content leg builds a typed detail LOCALLY, out of the edge's refusal token: + the token is the edge's, the sentence around it is this SDK's. So a detail is present + and the peer's message is still empty — the one case that separates "fill it from the + detail" from "fill it where the peer's own answer was decoded".""" + from ramp_sdk.errordetail import retrieval_auth_failure_detail + + err = CallError( + CallErrorKind.REFUSED, + "fetch content", + status=403, + reason="keyid_mismatch", + detail=retrieval_auth_failure_detail( + "ramp.v1.Edge", + "delivery refused: keyid_mismatch", + "RETRIEVAL_AUTH_FAILURE_REASON_KEYID_MISMATCH", + ), + ) + # The typed reason still reaches the caller — this is not about losing it. + assert err.detail is not None + assert err.peer_message == "" + + +@pytest.mark.parametrize("face", FACES, ids=_IDS) +def test_peer_message_stays_empty_for_a_pre_check_detail(face: Face) -> None: + """The registration pre-check is the same rule with no peer involved at all: the + request never left the process, so there is no remote party whose words this field + could hold. A detail is present because the schema failures are worth carrying; the + sentence around them is ours.""" + err = _refused_by_the_schema(face, _SCHEMA, {"trading_name": "Acme"}) + + assert err.detail is not None + assert err.peer_message == "" + + +@pytest.mark.parametrize("face", FACES, ids=_IDS) +def test_peer_message_carries_the_typed_reason_and_nothing_else(face: Face) -> None: + """The peer's own sentence reaches a caller as a VALUE, and only when the answer + carried a typed reason: transport-synthesised text is the transport's, not the + peer's.""" + typed = Recorder( + { + "code": "failed_precondition", + "message": "envelope prose", + "details": [ + { + "type": "ramp.v1.ErrorDetail", + "debug": { + "domain": "ramp.v1.ExchangeService", + "message": "terms have moved", + "registrationFailure": { + "reason": "REGISTRATION_FAILURE_REASON_TERMS_DIGEST_STALE" + }, + }, + } + ], + }, + 412, + ) + with pytest.raises(CallError) as caught: + face.run(face.client(_account_config(), typed).register({"exchange": "exchange.test"})) + assert caught.value.peer_message == "terms have moved" + # The token stays a token: prose never leaks into the field a caller branches on. + assert " " not in (caught.value.reason or "") + + untyped = Recorder({"code": "unavailable", "message": "draining"}, 503) + with pytest.raises(CallError) as caught: + face.run( + face.client(_account_config(), untyped).get_account_status( + {"exchange": "exchange.test"} + ) + ) + assert caught.value.peer_message == "" + + +# --------------------------------------------------------------------------- +# The requirements reader +# --------------------------------------------------------------------------- + + +def _manifest_reader(body: str) -> tuple[WellKnownRequirementsReader, list[int]]: + hits = [0] + + def respond(_request: httpx.Request) -> httpx.Response: + hits[0] += 1 + return httpx.Response(200, content=body.encode(), headers={"content-type": "application/json"}) + + http = httpx.Client(transport=httpx.MockTransport(respond)) + return WellKnownRequirementsReader(scheme="http", http=http), hits + + +def _manifest(**extra: Any) -> str: + doc: dict[str, Any] = {"role": "ROLE_EXCHANGE", "endpoint": _ENDPOINT} + doc.update(extra) + return json.dumps(doc) + + +def test_reader_fetches_afresh_on_every_read() -> None: + reader, hits = _manifest_reader(_manifest(terms_digest=_DIGEST)) + assert reader.resolve_registration_requirements("exchange.test").terms_digest == _DIGEST + reader.resolve_registration_requirements("exchange.test") + assert hits[0] == 2 + + +def test_reader_refuses_a_manifest_that_is_not_an_exchange() -> None: + for body in ( + _manifest(role="ROLE_BROKER"), + json.dumps({"endpoint": _ENDPOINT}), + ): + reader, _ = _manifest_reader(body) + with pytest.raises(ManifestNotExchangeError): + reader.resolve_registration_requirements("exchange.test") + + +def test_reader_refuses_before_dialling() -> None: + reader, hits = _manifest_reader(_manifest()) + with pytest.raises(ValueError): + reader.resolve_registration_requirements("exchange.test/path") + assert hits[0] == 0 + + def respond(_request: httpx.Request) -> httpx.Response: # pragma: no cover - never called + raise AssertionError("the allow overlay must run before the fetch") + + blocked = WellKnownRequirementsReader( + scheme="http", http=httpx.Client(transport=httpx.MockTransport(respond)), allow=lambda _d: False + ) + with pytest.raises(ExchangeNotPermittedError): + blocked.resolve_registration_requirements("exchange.test") + + +def test_reader_measures_the_schema_over_the_served_bytes() -> None: + """The cap is defined over the bytes AS SERVED. A schema padded past it on the wire + that would minify under it must be refused here exactly as the Exchange refuses it, + or a client pre-checks against a schema nobody enforces.""" + padded = '{"type":"object",' + " " * 20_000 + '"title":"x"}' + body = '{"role":"ROLE_EXCHANGE","account_registration":{"data_schema":' + padded + "}}" + reader, _ = _manifest_reader(body) + got = reader.resolve_registration_requirements("exchange.test") + assert got.verdict == "too_large" + assert got.schema is None + + +# The default reader is built ONCE per client, and closed with it. +# +# Its transport is the SSRF-guarded one, so a reader built per call opens a connection +# pool per registration and closes none — measured before this was fixed: three calls, +# three clients, none closed. Go resolves the same default once at NewClient; both faces +# here do it in their constructor, which is the same tier. +# +# Directly observable in Python, unlike TypeScript, because the client owns the transport +# and has a close() to prove it with; the TS side pins the same invariant structurally. +def test_client_builds_one_requirements_transport_and_closes_it() -> None: + from ramp_sdk.sync import Client as SyncClient + + cfg = _config(endpoint_resolver=_Resolver()) + assert cfg.registration_requirements is None + client = SyncClient(cfg, http=httpx.Client(transport=httpx.MockTransport( + lambda _r: httpx.Response(200, json={"ver": "1.0", "billing_ref": "acct-1"}) + ))) + try: + # Filled in once, on a COPY: a caller may hold its config and reuse it. + assert client._config.registration_requirements is not None + assert cfg.registration_requirements is None + owned = client._requirements_http + assert owned is not None + assert not owned.is_closed + finally: + client.close() + assert owned.is_closed + + +def test_client_leaves_an_injected_reader_alone() -> None: + """A reader the caller injected carries a transport the caller owns, so the client + builds nothing and closes nothing — the same rule the two RPC legs follow.""" + from ramp_sdk.sync import Client as SyncClient + + injected = _Requirements(RegistrationRequirements()) + client = SyncClient(_account_config(injected), http=httpx.Client( + transport=httpx.MockTransport(lambda _r: httpx.Response(200, json={"ver": "1.0"})) + )) + try: + assert client._requirements_http is None + assert client._config.registration_requirements is injected + finally: + client.close() diff --git a/sdk/python/tests/test_api_surface_parity.py b/sdk/python/tests/test_api_surface_parity.py index 3f3668f0..33a0c896 100644 --- a/sdk/python/tests/test_api_surface_parity.py +++ b/sdk/python/tests/test_api_surface_parity.py @@ -89,8 +89,11 @@ # in the same change — 19 undocumented divergences became 10 documented ones and 9 # exported Python symbols; that trade is the one sanctioned growth shape — every entry # must arrive with its reason, never bare), then shrunk 16 -> 14 by resolving the two -# ErrUnknownKey gaps, then grew 14 -> 16 under the sanctioned shape above. -BASELINE_ALLOWLIST = 16 +# ErrUnknownKey gaps, then grew 14 -> 16 under the sanctioned shape above, then 16 -> 17 +# under it again: the registration-requirements reader's Go factory folds into the Python +# class constructor exactly as every other NewX does, and its reason is that same +# recorded one. One entry, one already-recorded class, and the bump is the review tell. +BASELINE_ALLOWLIST = 17 # HARD ZERO (was a shrink-only ratchet at 19) — undocumented TS-present / Python-null # gaps. The PRESENCE check skips nulls, so absent this ceiling a NEW Python-null gap diff --git a/sdk/python/tests/test_client.py b/sdk/python/tests/test_client.py index 0890dd18..1d393d77 100644 --- a/sdk/python/tests/test_client.py +++ b/sdk/python/tests/test_client.py @@ -570,6 +570,29 @@ def resolve_endpoint(self, host: str) -> str: # noqa: ARG002 assert flaky.value.kind is CallErrorKind.UNREACHABLE +@pytest.mark.parametrize("face", FACES, ids=_IDS) +def test_an_unrelated_value_error_from_a_resolver_is_not_a_verdict(face: Face) -> None: + """The invalid-host refusal is a bare ValueError in this port, and the routing leg + once matched that TYPE. json.JSONDecodeError subclasses ValueError, so an injected + resolver that could not parse a document had its transport failure reported as a + permanent verdict — where the oracle and TypeScript both call it retryable. The + refusal is recognised by its wording now, through the one predicate the account leg + also uses. + """ + + class Unparsed: + def resolve_endpoint(self, host: str) -> str: # noqa: ARG002 + raise ValueError("expecting value: line 1 column 1 (char 0)") + + with pytest.raises(CallError) as caught: + face.run( + face.client(_config(endpoint_resolver=Unparsed()), Recorder({})).report_usage( + {"exchange": "issuer.test"} + ) + ) + assert caught.value.kind is CallErrorKind.UNREACHABLE + + @pytest.mark.parametrize("face", FACES, ids=_IDS) def test_an_unaccepted_manifest_version_is_not_sent(face: Face) -> None: """A manifest whose version the reader refuses is a VERDICT: the manifest was @@ -682,7 +705,7 @@ def respond(request: httpx.Request) -> httpx.Response: @pytest.mark.parametrize("face", FACES, ids=_IDS) def test_an_edge_refusal_carries_its_typed_reason(face: Face) -> None: def respond(request: httpx.Request) -> httpx.Response: # noqa: ARG001 - return httpx.Response(403, json={"error": "denied", "reason": "url_expired"}) + return httpx.Response(403, json={"error": "denied", "reason": "expired"}) transport = httpx.MockTransport(respond) config = _config() @@ -697,7 +720,7 @@ def respond(request: httpx.Request) -> httpx.Response: # noqa: ARG001 err = excinfo.value assert err.kind is CallErrorKind.REFUSED - assert err.reason_of() == "url_expired" + assert err.reason_of() == "expired" assert err.detail is not None assert ( err.detail.retrieval_auth_failure.reason.value diff --git a/sdk/python/tests/test_client_request_parity.py b/sdk/python/tests/test_client_request_parity.py index 91ceb956..b5dc4902 100644 --- a/sdk/python/tests/test_client_request_parity.py +++ b/sdk/python/tests/test_client_request_parity.py @@ -28,6 +28,7 @@ from conftest import GO_CONNECT_TESTDATA, load_json from ramp_sdk.client import BrokerClient, CatalogClient, Client, ClientConfig +from ramp_sdk.resolvers import RegistrationRequirements from ramp_sdk.core import Mode, StaticOfferKeyResolver, Verifier, sign_offer_jcs from ramp_sdk.idempotency import validate_idempotency_key from ramp_sdk.signing_transport import SigningTransport @@ -64,9 +65,21 @@ def _config() -> ClientConfig: requester=_REQUESTER, signer=SigningTransport(signer_seed=_AGENT_SEED, keyid="agent.v1"), endpoint_resolver=_Resolver(), + # Register reads an Exchange's published requirements before it signs. What this + # corpus records is the path and the envelope, so the read is stubbed to an + # Exchange that publishes neither a digest nor a schema — the ordinary + # pass-through case. Inert for every other verb. + registration_requirements=_PublishesNothing(), ) +class _PublishesNothing: + """An Exchange that publishes neither a terms digest nor a registration schema.""" + + def resolve_registration_requirements(self, _exchange: str) -> RegistrationRequirements: + return RegistrationRequirements() + + def _verified_offer() -> Any: offer: dict[str, Any] = { "offer_id": "offer-good", @@ -118,6 +131,13 @@ def _call(name: str) -> httpx.Request: asyncio.run( Client(config, http=http).execute(_verified_offer(), idempotency_key=_PINNED) ) + elif name.startswith("register"): + register: dict[str, Any] = {"exchange": _ISSUER} + if name == "register_caller_ver_wins": + register["ver"] = "9.9" + asyncio.run(Client(config, http=http).register(register)) + elif name == "get_account_status": + asyncio.run(Client(config, http=http).get_account_status({"exchange": _ISSUER})) elif name == "resolve": asyncio.run(BrokerClient(config, http=http).resolve({})) elif name.startswith("push_resources"): diff --git a/sdk/python/tests/test_guards_resolvers_transport_defaults.py b/sdk/python/tests/test_guards_resolvers_transport_defaults.py new file mode 100644 index 00000000..0142aef1 --- /dev/null +++ b/sdk/python/tests/test_guards_resolvers_transport_defaults.py @@ -0,0 +1,157 @@ +"""Which transport each resolver takes when the caller injects none. + +Why this is a test and not a sentence +------------------------------------- +The rule is that a resolver's default follows its URL's PROVENANCE: a fixed, +operator-chosen address takes the plain client, a host another party named takes the +guarded one. That rule is written down in three places — the Go oracle's +``WellKnownOptions.HTTP`` comment, ``docs/design-history.md``, and the published threat +model, which states it per language in a table. + +It has now drifted from the code twice. The scheme guard was added to the dial and not +walked back through its callers, so a doc sentence beginning "every fetch" stayed standing +after it stopped being true; and this port mirrored the oracle's options struct without +mirroring the rule, then explained the result in a docstring that had also stopped being +true. Prose cannot detect either. A socket can. + +SO: CHANGING ANY ROW BELOW OBLIGES AN EDIT TO THE PUBLISHED THREAT MODEL +(``website/src/content/docs/security/threat-model.mdx``), which names these postures per +language. That is the whole point of the gate — the failure is the reminder. + +One row records a gap, not a decision +------------------------------------- +``endpoint resolver`` REACHES here and REFUSES in Go. That is not a considered +divergence: the host is an ``Offer.exchange`` domain, so by the rule above it belongs on +the guarded transport in every language, and this port has not caught up. It is tracked +separately, and the threat model names it. Do not read this row as endorsement — read it +as the thing that will fail, loudly and on purpose, in the change that fixes it. + +What is not here +---------------- +The WBA directory resolver's guarded default is pinned already, by +``test_resolvers_wba_ssrf.py`` and its Go and TypeScript counterparts; its guard is +additionally not opt-outable, which those cover and this would not. Asserting it twice +would be one behaviour with two owners. + +This is deliberately NOT a shared corpus. A ``*-vectors.json`` has to be replayed by all +three languages against the same expectations, and the expectations here legitimately +DIFFER per language — that difference is the subject. +""" + +from __future__ import annotations + +import http.server +import threading +from collections.abc import Callable +from typing import Any + +import pytest + +from ramp_sdk.resolvers import ( + WellKnownEndpointResolver, + WellKnownKeyResolver, + WellKnownRequirementsReader, +) + + +def _dial_observed(exercise: Callable[[str], Any]) -> bool: + """Whether a resolver reached the loopback origin. + + A guarded default refuses the reserved address before the request leaves the process, + so "the handler ran" is the observable that separates the two transports. + """ + reached = False + + class _Handler(http.server.BaseHTTPRequestHandler): + def do_GET(self) -> None: # BaseHTTPRequestHandler contract + nonlocal reached + reached = True + # Anchored to the host that served it, so the endpoint face's own + # same-host rule cannot be what refuses. + body = ( + b'{"role":"ROLE_EXCHANGE","endpoint":"https://' + + self.headers.get("Host", "").encode() + + b'","keys":[]}' + ) + self.send_response(200) + self.send_header("Content-Type", "application/json") + self.send_header("Content-Length", str(len(body))) + self.end_headers() + self.wfile.write(body) + + def log_message(self, *_args: object) -> None: + return + + server = http.server.ThreadingHTTPServer(("127.0.0.1", 0), _Handler) + port = server.server_address[1] + threading.Thread(target=server.serve_forever, daemon=True).start() + try: + # The exception is deliberately swallowed: a guarded refusal and a happy answer + # are both expected outcomes, and ``reached`` is what says which. + try: + exercise(f"127.0.0.1:{port}") + except Exception: # noqa: BLE001 - the outcome under test is the dial, not the error + pass + finally: + server.shutdown() + server.server_close() + return reached + + +def _key(host: str) -> Any: + return WellKnownKeyResolver(f"http://{host}/.well-known/ramp.json").resolve("ex.v1") + + +def _endpoint(host: str) -> Any: + return WellKnownEndpointResolver(scheme="http").resolve_endpoint(host) + + +def _requirements(host: str) -> Any: + return WellKnownRequirementsReader(scheme="http").resolve_registration_requirements(host) + + +#: ``reaches`` is True where the default is the PLAIN client. +_ROWS = [ + pytest.param( + _key, + True, + "its URL is a fixed, operator-chosen JWKS address — nobody else can point it, " + "and an on-prem directory may legitimately be private", + id="key resolver dials plain", + ), + pytest.param( + _endpoint, + True, + "its host is an Offer.exchange domain, so another party chose the address and the " + "rule says guarded. Go already refuses here; this port has not caught up", + id="endpoint resolver dials plain - a tracked gap, not a decision", + ), + pytest.param( + _requirements, + False, + "its host is a RegisterRequest.exchange domain, named by the caller per call", + id="requirements reader dials guarded", + ), +] + + +@pytest.mark.parametrize(("exercise", "reaches", "why"), _ROWS) +def test_resolver_transport_defaults( + exercise: Callable[[str], Any], + reaches: bool, + why: str, + monkeypatch: pytest.MonkeyPatch, +) -> None: + # Both guards are env-driven, so the flags are cleared rather than inherited from + # whatever shell ran the suite: without this a developer with SKIP_SSRF set sees the + # guarded rows fail for a reason that has nothing to do with the defaults. + for name in ("SKIP_SSRF", "ALLOW_INSECURE", "HTTP_PROXY", "HTTPS_PROXY"): + monkeypatch.delenv(name, raising=False) + + got = _dial_observed(exercise) + verb = {True: "reached", False: "refused"} + assert got == reaches, ( + f"default transport {verb[got]} the loopback origin, want {verb[reaches]} — {why}. " + "If this change is intended, the per-language table in the published threat model " + "has to change with it." + ) diff --git a/sdk/python/tests/test_guards_window_adoption.py b/sdk/python/tests/test_guards_window_adoption.py index 54fa5a2f..52739801 100644 --- a/sdk/python/tests/test_guards_window_adoption.py +++ b/sdk/python/tests/test_guards_window_adoption.py @@ -21,9 +21,13 @@ # The pre-migration inline form the sign site must no longer contain. _FORBIDDEN = re.compile(r"created\s*\+\s*self\._ttl_sec") +# The sign site must CALL an injected Window. A caller-supplied window may override +# the one the transport was built with — the client surfaces that knob at the tier Go +# and TypeScript surface it at — so the call reads `(window or self._window)()` there; +# what stays required is that `self._window` is the fallback rather than arithmetic. _REQUIRED = ( re.compile(r"from\s+ramp_sdk\.window\s+import"), - re.compile(r"self\._window\(\)"), + re.compile(r"self\._window\)?\(\)"), ) @@ -56,10 +60,19 @@ def test_meta_negative_passes_window_form(self) -> None: good = "from ramp_sdk.window import Window, clock_window\ncreated, expires = self._window()" assert _eval(good) == [] + def test_meta_negative_passes_caller_override_form(self) -> None: + """A caller-supplied window overriding the transport's own is still the injected + Window: the fallback is `self._window`, not an inline mint.""" + good = ( + "from ramp_sdk.window import Window, clock_window\n" + "created, expires = (window or self._window)()" + ) + assert _eval(good) == [] + def test_meta_would_be_missed_catches_reformatted_mint(self) -> None: reformatted = ( "from ramp_sdk.window import clock_window\n" - "self._window()\n" + "(window or self._window)()\n" "expires = created + self._ttl_sec\n" # extra spaces ) violations = _eval(reformatted) diff --git a/sdk/python/tests/test_parity_corpora_nonempty.py b/sdk/python/tests/test_parity_corpora_nonempty.py index df8d7333..ab7c2b38 100644 --- a/sdk/python/tests/test_parity_corpora_nonempty.py +++ b/sdk/python/tests/test_parity_corpora_nonempty.py @@ -120,6 +120,16 @@ def _whole(doc: Any) -> Any: _vectors, "client-request", ), + ( + GO_CONNECT_TESTDATA / "synthesized-detail-vectors.json", + lambda d: d["registration_precheck"], + "synthesized-detail-precheck", + ), + ( + GO_CONNECT_TESTDATA / "synthesized-detail-vectors.json", + lambda d: d["edge_refusal"], + "synthesized-detail-edge", + ), ( GO_RESOLVERS_TESTDATA / "content-fetch-vectors.json", _vectors, diff --git a/sdk/python/tests/test_synthesized_detail_parity.py b/sdk/python/tests/test_synthesized_detail_parity.py new file mode 100644 index 00000000..48ac5e60 --- /dev/null +++ b/sdk/python/tests/test_synthesized_detail_parity.py @@ -0,0 +1,139 @@ +"""Synthesized-detail parity (Python side) — replay of the shared Go-oracle corpus. + +Mirrors ``sdk/ts/tests/synthesized-detail.parity.test.ts`` and the Go leg +``sdk/go/connect/synthesized_detail_corpus_test.go``. + +Decoding a peer's ErrorDetail is already pinned elsewhere. These two details are +different: nothing on the wire says what they should contain, so the domain, the sentence +and the typed reason are AUTHORED — three times, in three languages, from three copies of +one decision. That is the shape that drifts silently, and it had: this port derived each +edge reason's enum name by uppercasing the token, which is right for two of the eleven +tokens the protocol records and wrong for the rest, so nine real edge refusals carried a +typed reason in Go and none here. + +Field errors are compared by PATH and never by text. The constraint prose comes from a +different validator library in each language and the contract calls it non-authoritative, +so pinning it here would pin an accident. +""" + +from __future__ import annotations + +from typing import Any + +import httpx +import pytest +from test_client import FACES, Face, _IDS, _config # noqa: PLC2701 + +from conftest import GO_CONNECT_TESTDATA, load_json +import ramp_sdk.sync as sync_client +from ramp_sdk.client import Client +from ramp_sdk.client.errors import CallError, CallErrorKind +from ramp_sdk.errordetail import REASON_FIELDS, reason +from ramp_sdk.regschema import compile_registration_schema +from ramp_sdk.resolvers import RegistrationRequirements + +_CORPUS = load_json(GO_CONNECT_TESTDATA / "synthesized-detail-vectors.json") +_PRECHECK = _CORPUS["registration_precheck"] +_EDGE = _CORPUS["edge_refusal"] + +_ENDPOINT = "https://exchange.test" + + +class _Resolver: + def resolve_endpoint(self, _host: str) -> str: + return _ENDPOINT + + +class _Requirements: + def __init__(self, reqs: RegistrationRequirements) -> None: + self._reqs = reqs + + def resolve_registration_requirements(self, _exchange: str) -> RegistrationRequirements: + return self._reqs + + +def _refuse(request: httpx.Request) -> httpx.Response: # noqa: ARG001 + raise AssertionError("the pre-check refusal must never reach the wire") + + +def _assert_recorded_detail(err: CallError, vector: dict[str, Any]) -> None: + """The detail matches what the oracle recorded for this row.""" + detail = err.detail + assert detail is not None + assert (detail.domain or "") == vector["domain"] + assert (detail.message or "") == vector["message"] + # Exactly the recorded oneof member is populated, and only it. + populated = [f for f in REASON_FIELDS if getattr(detail, f, None) is not None] + assert populated == [vector["reason_field"]] + got = reason(detail) + assert got is not None + assert got.value == vector["reason_enum"] + + +@pytest.mark.parametrize("face", FACES, ids=_IDS) +@pytest.mark.parametrize("vector", _PRECHECK, ids=[v["name"] for v in _PRECHECK]) +def test_the_registration_pre_check_detail_matches_the_oracle( + vector: dict[str, Any], face: Face +) -> None: + schema, verdict = compile_registration_schema(vector["schema"].encode()) + assert verdict == "accepted", "the row's own schema does not compile" + config = _config( + endpoint_resolver=_Resolver(), + registration_requirements=_Requirements( + RegistrationRequirements(schema=schema, verdict=verdict) + ), + ) + transport = httpx.MockTransport(_refuse) + client = ( + Client(config, http=httpx.AsyncClient(transport=transport)) + if face.name == "async" + else sync_client.Client(config, http=httpx.Client(transport=transport)) + ) + + with pytest.raises(CallError) as caught: + face.run( + client.register( + {"exchange": "exchange.test", "registration_data": vector["data"]} + ) + ) + + err = caught.value + assert err.kind is CallErrorKind.MALFORMED + _assert_recorded_detail(err, vector) + failure = err.detail.registration_failure # type: ignore[union-attr] + assert [f.path or "" for f in failure.field_errors or []] == vector["field_error_paths"] + + +@pytest.mark.parametrize("face", FACES, ids=_IDS) +@pytest.mark.parametrize("vector", _EDGE, ids=[v["name"] for v in _EDGE]) +def test_the_content_leg_detail_matches_the_oracle( + vector: dict[str, Any], face: Face +) -> None: + token = vector["reason_token"] + + def respond(request: httpx.Request) -> httpx.Response: # noqa: ARG001 + return httpx.Response(403, json={"error": "denied", "reason": token}) + + transport = httpx.MockTransport(respond) + config = _config() + client = ( + Client(config, http=httpx.AsyncClient(transport=transport)) + if face.name == "async" + else sync_client.Client(config, http=httpx.Client(transport=transport)) + ) + + with pytest.raises(CallError) as caught: + face.run(client.fetch("https://edge.test/doc")) + + err = caught.value + # The raw token reaches the caller either way; only the typed reason is at stake, so a + # row that records none must still carry the refusal. + assert err.reason_of() == token + + if vector["reason_field"] == "": + # The protocol records no reason for this token, or records two and so cannot say + # which check ran. Promoting one would attribute a failure the wire never + # attributed. + assert err.detail is None + return + _assert_recorded_detail(err, vector) diff --git a/sdk/ts/client/content.ts b/sdk/ts/client/content.ts index 1cad05fa..543d5d77 100644 --- a/sdk/ts/client/content.ts +++ b/sdk/ts/client/content.ts @@ -353,15 +353,43 @@ async function edgeRefusal( ); } +/** The edge's refusal token, mapped to its typed reason. + * + * The two vocabularies are separate on purpose — the edge is a code-capable worker with + * no protobuf runtime, so it answers a string — and `ramp.proto` records which token each + * enum value stands for, beside the value. This table IS that record; it is not derived + * from the enum's own spelling, and deriving it would be wrong: `expired` names + * URL_EXPIRED, `pop_expired` names PROOF_EXPIRED, and neither is the enum suffix + * lowercased. A port that computed the name instead of reading the record typed two of + * the eleven tokens and silently dropped the rest. + * + * `missing_sig` is deliberately ABSENT rather than guessed at: both checkers emit it — + * the signed-URL check for a missing `sig` parameter and the proof check for a missing + * Signature header — and the enum has a distinct value for each, so the body does not say + * which ran. An unmapped token still reaches the caller as the raw refusal string; only + * the typed reason is withheld, which is the honest outcome when the wire cannot say + * which failure occurred. The edge's parse-level tokens have no enum value at all. */ +const EDGE_REASON_TOKENS: Readonly> = { + // Signed-URL checks. + expired: "RETRIEVAL_AUTH_FAILURE_REASON_URL_EXPIRED", + missing_exp: "RETRIEVAL_AUTH_FAILURE_REASON_URL_EXPIRY_MISSING", + signature_mismatch: "RETRIEVAL_AUTH_FAILURE_REASON_URL_SIGNATURE_MISMATCH", + // Proof-of-possession checks. + missing_agent_key: "RETRIEVAL_AUTH_FAILURE_REASON_AGENT_KEY_MISSING", + keyid_mismatch: "RETRIEVAL_AUTH_FAILURE_REASON_KEYID_MISMATCH", + thumbprint_mismatch: "RETRIEVAL_AUTH_FAILURE_REASON_THUMBPRINT_MISMATCH", + pop_missing_created: "RETRIEVAL_AUTH_FAILURE_REASON_PROOF_CREATED_MISSING", + pop_missing_exp: "RETRIEVAL_AUTH_FAILURE_REASON_PROOF_EXPIRY_MISSING", + pop_expired: "RETRIEVAL_AUTH_FAILURE_REASON_PROOF_EXPIRED", + pop_sig_invalid: "RETRIEVAL_AUTH_FAILURE_REASON_PROOF_SIGNATURE_INVALID", +}; + // retrievalAuthFailureDetailOrUndefined promotes an edge token to the typed reason when -// it names one of the protocol's own values, and answers undefined otherwise. The edge's -// vocabulary and the protocol's are deliberately separate, and this is the single place -// they meet. +// the protocol records one for it, and answers undefined otherwise. The edge's vocabulary +// and the protocol's are deliberately separate, and this is the single place they meet. function retrievalAuthFailureDetailOrUndefined(token: string) { - const upper = token.toUpperCase(); - const named = upper.startsWith("RETRIEVAL_AUTH_FAILURE_REASON_") - ? upper - : `RETRIEVAL_AUTH_FAILURE_REASON_${upper}`; + const named = EDGE_REASON_TOKENS[token]; + if (named === undefined) return undefined; try { return retrievalAuthFailureDetail( EDGE_ERROR_DOMAIN, diff --git a/sdk/ts/client/errors.ts b/sdk/ts/client/errors.ts index 129f53a3..0d30fba5 100644 --- a/sdk/ts/client/errors.ts +++ b/sdk/ts/client/errors.ts @@ -49,6 +49,11 @@ export interface CallErrorInit { reason?: string; /** The typed protocol reason when there is one. */ detail?: ErrorDetail; + /** The developer message the peer put on its TYPED reason, when it sent one. + * Deliberately not filled from the transport envelope: an answer that did not + * come from a RAMP service carries no message of its own, and the text a + * transport synthesizes for one is that transport's rather than the peer's. */ + peerMessage?: string; /** The underlying failure, kept so a caller can still reach a custody or resolver * sentinel after the failure has been classified here. */ cause?: unknown; @@ -66,6 +71,17 @@ export class RampCallError extends Error { readonly status: number | undefined; readonly reason: string | undefined; readonly detail: ErrorDetail | undefined; + /** The peer's own developer message, taken from its typed reason. Empty when the + * answer carried none. + * + * A field rather than something to recover from `message`'s rendering, because a + * reason rendered into prose cannot be read back out without parsing it. It sits + * BESIDE `reason` rather than in it: `reason` is the peer's machine token. + * + * NON-AUTHORITATIVE and UNBOUNDED — the contract says both of the field it comes + * from. Branch on `kind` or on the typed reason, never on this text, and bound it + * before rendering it to a log line or an agent. */ + readonly peerMessage: string; constructor(init: CallErrorInit) { super(renderCallError(init), init.cause !== undefined ? { cause: init.cause } : undefined); @@ -75,6 +91,13 @@ export class RampCallError extends Error { this.status = init.status; this.reason = init.reason; this.detail = init.detail; + // Taken ONLY from what the decode site passed, never derived from `detail` + // here. A detail is not always the peer's: the content leg SYNTHESIZES one + // from the edge's refusal token, and its message is this SDK's own sentence + // with the token quoted into it. Deriving from the field would put the SDK's + // words in the field that claims to hold a remote party's, which is the one + // thing this field exists not to do. + this.peerMessage = init.peerMessage ?? ""; } /** diff --git a/sdk/ts/client/index.ts b/sdk/ts/client/index.ts index 244ad2f5..78ddb551 100644 --- a/sdk/ts/client/index.ts +++ b/sdk/ts/client/index.ts @@ -26,6 +26,7 @@ import { signRequestAcceptance, ACCEPTANCE_SIGNATURE_ALGORITHM, } from "../src/acceptance.ts"; +import { registrationFailureDetail } from "../src/errordetail.ts"; import { redactUserinfo } from "../src/host-ref.ts"; import { isBareDomain } from "../src/hosts.ts"; import { generateIdempotencyKey } from "../src/idempotency.ts"; @@ -35,10 +36,14 @@ import { DiscoveryResponseSchema, DisputeRequestSchema, DisputeResponseSchema, + GetAccountStatusRequestSchema, + GetAccountStatusResponseSchema, PushResourcesRequestSchema, PushResourcesResponseSchema, RefreshCatalogRequestSchema, RefreshCatalogResponseSchema, + RegisterRequestSchema, + RegisterResponseSchema, RemoveResourcesRequestSchema, RemoveResourcesResponseSchema, ResourceQuerySchema, @@ -50,8 +55,16 @@ import { } from "../../../gen/ts/wire/schemas.ts"; import { type Content, fetchContent } from "./content.ts"; import { malformed, notSent, RampCallError } from "./errors.ts"; +import { checkRegistrationData } from "../src/regschema.ts"; +import { + createWellKnownRequirementsReader, + ExchangeNotPermitted, + ManifestNotExchange, + ManifestUnusable, + type RegistrationRequirements, +} from "../resolvers/index.ts"; import type { EndpointResolver } from "./route.ts"; -import { vetExchangeEndpoint } from "./route.ts"; +import { isInvalidHostRefusal, vetExchangeEndpoint } from "./route.ts"; import { createUnarySend } from "./send.ts"; import { DEFAULT_CALL_TIMEOUT_MS, @@ -76,6 +89,35 @@ const CATALOG_SERVICE = "ramp.v1.CatalogService"; export const DEFAULT_PROOF_WINDOW_SEC = 30; /** Everything a client is built from. Every field is injected; the client owns none of it. */ +/** Reports what one Exchange asks of a registration. + * + * An interface for the same two reasons the endpoint seam is one: a test can drive a + * registration without standing up a manifest server, and this module has no way to + * accept a terms digest or a schema from configuration — the only way to skip the read + * is to set `terms_digest` on the request, where the signature covers it. + * + * An implementation MUST NOT serve the answer from a cache. The contract requires a + * registering client to read the digest from a freshly fetched manifest, so a cached one + * breaks the rule the field exists to record. + * + * An implementation's FAILURE decides how a caller is told to react, so it is part of the + * contract rather than an implementation detail. A failure that is a VERDICT — the domain + * is unusable, the deployment excludes it, the document served is not an Exchange's, or it + * is one this reader cannot use — MUST throw the resolver tier's ExchangeNotPermitted, + * ManifestNotExchange or ManifestUnusable, or the invalid-host error isBareHost raises; + * those surface as `not_sent`, which tells the caller not to retry. Anything else is read + * as a transport failure and reported as `unreachable`, i.e. worth retrying. An + * implementation that throws a bare error for a refusal therefore has its final answer + * retried indefinitely. + * + * ManifestUnusable is the one an implementation stricter than the SDK's own reaches for. + * That reader refuses three things and treats every other disappointment as absence or as + * a transport failure; one that validates the whole document, or refuses a version, is + * holding a final answer this seam would otherwise report as transient. */ +export interface RegistrationRequirementsReader { + resolveRegistrationRequirements(exchange: string): Promise; +} + export interface ClientOptions { /** The RFC 9421 request signer. Custody stays with the application — the SDK receives * a non-extractable CryptoKey and the keyid it signs under, never key bytes. */ @@ -109,6 +151,20 @@ export interface ClientOptions { /** Turns an offer's exchange domain into that Exchange's own advertised origin. Never * configuration — a usage report and a dispute go where the signed offer says. */ endpointResolver?: EndpointResolver; + /** Reports what one Exchange asks of a registration — the terms revision submitting + * one accepts, and the schema its registration_data must match. Defaults to the + * well-known reader over the SSRF-guarded transport, built once with this client: + * the domain comes off the request rather than from configuration, so it is the + * request-derived provenance that takes the guarded default. + * + * The reader it takes holds no document cache, and that is the point rather than an + * implementation detail: the contract requires a registering client to read the terms + * digest from a FRESHLY fetched manifest, so an implementation serving it from a + * cache breaks the rule the field exists to record. There is deliberately no option + * to supply a digest or a schema directly — a caller managing its own requirements + * sets `terms_digest` on the request, which suppresses the read and says so on the + * message the signature covers. */ + registrationRequirements?: RegistrationRequirementsReader; /** The WBA directory origin this client signs as. */ signatureAgent?: string; /** The RFC 9421 freshness window stamped on every outbound call. */ @@ -169,6 +225,13 @@ export interface Client { opts?: CallOptions, ): Promise; dispute(request: Record, opts?: CallOptions): Promise; + /** Create this agent's account at the Exchange the request names. Takes no + * CallOptions: the message carries no idempotency key, because registering again + * returns the same account handle. */ + register(request: Record): Promise; + /** Read whether this agent's account at the named Exchange is active. An empty + * `billing_ref` is a NORMAL answer — no account there yet. */ + getAccountStatus(request: Record): Promise; fetch(signedURL: string): Promise; } @@ -184,6 +247,11 @@ interface Resolved { verifier: Verifier; send: UnarySend; guardedSend: UnarySend; + /** The SEAM, not the concrete reader: an injected one and the default are the + * same thing to every caller below here. Resolved alongside the transports + * because the default holds a dispatcher, so building it per call would open + * one per registration and close none. */ + requirements: RegistrationRequirementsReader; signer: CallSigner | undefined; } @@ -211,6 +279,8 @@ function resolve(opts: ClientOptions): Resolved { verifier, send: opts.send ?? createUnarySend({ guarded: false }), guardedSend: opts.guardedSend ?? createUnarySend({ guarded: true }), + requirements: + opts.registrationRequirements ?? createWellKnownRequirementsReader(), signer, }; } @@ -255,6 +325,8 @@ export function createClient(baseURL: string, options: ClientOptions = {}): Clie execute: (offer, opts) => execute(r, baseURL, offer, opts ?? {}), reportUsage: (report, opts) => reportUsage(r, report, opts ?? {}), dispute: (request, opts) => dispute(r, request, opts ?? {}), + register: (request) => register(r, request), + getAccountStatus: (request) => getAccountStatus(r, request), fetch: (signedURL) => fetchVerb(r, signedURL), }; } @@ -661,6 +733,174 @@ async function dispute( return parseMessage(op, raw, DisputeResponseSchema); } +// --------------------------------------------------------------------------- +// The account-setup verbs +// +// They route like a usage report, not like discovery. An account is per-Exchange, +// and which Exchange is the agent's choice PER CALL: a target routinely arrives at +// runtime — a denial names where to register — rather than from configuration. So +// the destination is read off the request's own `exchange` field and resolved +// through that Exchange's own manifest, over the guarded leg. +// +// Neither message carries an idempotency key, so neither verb takes CallOptions. +// --------------------------------------------------------------------------- + +/** The ErrorDetail domain for a refusal THIS CLIENT computed, before anything was + * sent. It names the failing surface, which here is the client's own tier: the + * Exchange never saw the request, so naming it would attribute a local verdict to a + * party that reached none. The naming rule the value follows — a Service suffix for + * an RPC service that exists in the contract, a bare noun for a tier that does not — + * is recorded on the Go oracle's edgeErrorDomain, beside EDGE_ERROR_DOMAIN's twin. */ +const CLIENT_ERROR_DOMAIN = "ramp.v1.Client"; + +/** + * register creates the calling agent's account at the Exchange the request names. + * + * The caller's identity is the request SIGNATURE. Nothing in the message says who is + * registering, and the business payload is not an identity claim. + * + * Four bounds on `registration_data` are checked before anything is signed, in the order + * the contract fixes, because a limit that exists to stop work belongs before the work it + * would stop — including before the manifest read. + * + * `terms_digest` is filled only when the caller left it ABSENT, from a freshly fetched + * manifest, and the payload is pre-checked against the schema that manifest publishes. A + * caller that sets the field is managing its own requirements and gets neither. + * + * A schema this SDK refuses never becomes a local veto: refusing locally and declining to + * send would turn a rule about reading a third party's document into a denial of service + * against the caller's own user, so an unusable schema is skipped and the Exchange + * decides. A usable schema the payload fails is the pre-check working, and that request is + * refused here with the offending members named. + */ +async function register( + r: Resolved, + request: Record, +): Promise { + const op = "register"; + const sent = stampVer(op, request); + requireRecipient(op, stringField(sent, "exchange")); + const verdict = checkRegistrationData( + (sent.registration_data ?? null) as Record | null, + ); + if (verdict !== "accepted") { + throw malformed(op, new Error(`registration_data: ${verdict}`)); + } + if (sent.terms_digest === undefined || sent.terms_digest === null) { + await applyRegistrationRequirements(r, op, sent); + } + const endpoint = await vetExchangeEndpoint( + r.opts.endpointResolver, + stringField(sent, "exchange"), + op, + ); + validateRequest(op, sent, RegisterRequestSchema, r.opts.validation ?? "strict"); + const raw = await call(r, op, endpoint, EXCHANGE_SERVICE, "Register", sent, true); + return parseMessage(op, raw, RegisterResponseSchema); +} + +/** + * getAccountStatus reports whether the calling agent's account at the named Exchange is + * active. + * + * The request carries no field identifying the caller — the Exchange resolves the account + * from the verified signature — so `exchange` is the only thing that says which account is + * being asked about. An empty `billing_ref` in the answer is a NORMAL answer: no account + * there yet. + * + * A caveat worth knowing before calling this in a loop. The request has no varying field, + * so two calls to the same Exchange inside one wall-clock second sign IDENTICAL bytes, and + * a peer screening replays on (key id, signature) refuses the second. This verb does not + * choose the freshness window for you, because a window is one instance per client rather + * than per call: pass `monotonicWindow` as `signWindow` when repeat calls are expected. + */ +async function getAccountStatus( + r: Resolved, + request: Record, +): Promise { + const op = "get account status"; + const sent = stampVer(op, request); + requireRecipient(op, stringField(sent, "exchange")); + const endpoint = await vetExchangeEndpoint( + r.opts.endpointResolver, + stringField(sent, "exchange"), + op, + ); + validateRequest(op, sent, GetAccountStatusRequestSchema, r.opts.validation ?? "strict"); + const raw = await call(r, op, endpoint, EXCHANGE_SERVICE, "GetAccountStatus", sent, true); + return parseMessage(op, raw, GetAccountStatusResponseSchema); +} + +/** + * applyRegistrationRequirements reads what the Exchange asks of a registration and applies + * it to the request being built. + * + * A failed READ refuses the registration rather than sending without a digest. Guessing + * here is not the cautious option: an Exchange that publishes a digest refuses a + * registration that omits one, so sending anyway trades a local failure the caller can act + * on for a remote one it cannot. + */ +async function applyRegistrationRequirements( + r: Resolved, + op: string, + sent: Record, +): Promise { + let reqs: RegistrationRequirements; + try { + reqs = await r.requirements.resolveRegistrationRequirements( + stringField(sent, "exchange"), + ); + } catch (err) { + // A value this deployment or the Exchange refused is FINAL; anything else is a + // transport failure worth retrying. The same split the routing tier makes, and + // the same causes: a value that is not a host will not become one on a later + // attempt either, and a document that arrived unusable arrives unusable again. + // The verb's own recipient check runs the host rule first, and the SDK's own + // reader never throws ManifestUnusable at all — so these reach here only through + // an INJECTED reader, and classifying its refusal as retryable would have a + // caller retry a verdict. + if ( + err instanceof ExchangeNotPermitted || + err instanceof ManifestNotExchange || + err instanceof ManifestUnusable || + isInvalidHostRefusal(err) + ) { + throw notSent(op, err); + } + throw new RampCallError({ kind: "unreachable", op, cause: err }); + } + if (reqs.termsDigest !== undefined) { + sent.terms_digest = reqs.termsDigest; + } + // A null validator means "nothing to enforce", which is the behaviour the contract + // requires both when the Exchange publishes no schema and when it publishes one this + // SDK refused. One branch, deliberately. + const fails = reqs.schema?.validate(sent.registration_data ?? {}) ?? []; + if (fails.length > 0) { + // An empty path addresses the whole object, which is how a missing required + // member and every other whole-object failure is reported. Rendering a bare + // ": ..." there would read as a member with no name. + const named = fails.map((f) => (f.path ? `${f.path}: ${f.error}` : f.error)).join("; "); + // The failures travel as a typed detail, not only as prose. An Exchange attaches + // this same list when it refuses the same payload, so a consumer that renders one + // refusal renders both, and nothing has to parse the members back out of a + // sentence. + throw new RampCallError({ + kind: "malformed", + op, + cause: new Error( + `registration_data does not match the schema ${stringField(sent, "exchange")} publishes: ${named}`, + ), + detail: registrationFailureDetail( + CLIENT_ERROR_DOMAIN, + "registration_data does not match the published data_schema", + "REGISTRATION_FAILURE_REASON_INVALID_REGISTRATION_DATA", + fails, + ), + }); + } +} + /** * fetch retrieves the content a signed delivery URL names, presenting proof of possession * of the agent key that URL is bound to. @@ -794,6 +1034,10 @@ function isRecord(v: unknown): v is Record { // --------------------------------------------------------------------------- /** The answer to a catalog push: accepted/rejected counts and the warnings the accepted terms carry. */ +export type RegisterResponse = z.infer; + +export type GetAccountStatusResponse = z.infer; + export type PushResourcesResponse = z.infer; /** The answer to a catalog removal. */ export type RemoveResourcesResponse = z.infer; diff --git a/sdk/ts/client/route.ts b/sdk/ts/client/route.ts index 7067d75b..cc6c8def 100644 --- a/sdk/ts/client/route.ts +++ b/sdk/ts/client/route.ts @@ -133,6 +133,22 @@ export async function vetExchangeEndpoint( return endpoint; } +/** Whether a cause is the invalid-host refusal. + * + * Matched on its wording rather than its type. The host helpers are L1 and deliberately + * raise a bare error for it — a typed InvalidHost would cross the helpers-versus-resolvers + * sentinel split the SDK documents — so the shared prefix is what there is to match. + * + * Exported because TWO injectable readers can raise it, the endpoint resolver and the + * registration-requirements reader, and both have to call it final for the same reason. A + * second copy of the prefix is exactly how the two would come to disagree. */ +export function isInvalidHostRefusal(cause: unknown): boolean { + return ( + cause instanceof Error && + cause.message.startsWith("hosts: reference is not a usable host") + ); +} + // isVerdict tells the resolver's final answers from its transient ones. The invalid-host // error is in the set because the resolver checks the host itself too, and a value that // is not a host will not become one on a later attempt. This module checks it before @@ -145,14 +161,5 @@ function isVerdict(cause: unknown): boolean { ) { return true; } - // The invalid-host refusal is matched on its wording rather than its type. The host - // helpers are L1 and deliberately raise a bare error for it — a typed InvalidHost - // would cross the helpers-versus-resolvers sentinel split the SDK documents — so the - // shared prefix is what there is to match. It is only reachable through an INJECTED - // resolver: the bare-host check above runs first, so the SDK's own resolver can never - // be handed a value that fails it here. - return ( - cause instanceof Error && - cause.message.startsWith("hosts: reference is not a usable host") - ); + return isInvalidHostRefusal(cause); } diff --git a/sdk/ts/client/transport.ts b/sdk/ts/client/transport.ts index 7537bd29..7e79fd3e 100644 --- a/sdk/ts/client/transport.ts +++ b/sdk/ts/client/transport.ts @@ -405,6 +405,12 @@ function connectEnvelopeError( // than the class reads the typed detail. reason: code, ...(detail !== null ? { detail } : {}), + // The one site that fills peerMessage, because this is the one site holding a + // detail the PEER emitted. Every other CallError leaves it empty — including + // the content leg, whose detail this SDK writes itself. + ...(detail?.message !== undefined && detail.message !== "" + ? { peerMessage: detail.message } + : {}), cause: typeof envelope["message"] === "string" ? envelope["message"] : undefined, }); diff --git a/sdk/ts/core/window.ts b/sdk/ts/core/window.ts index b9b559b9..a9b2e99c 100644 --- a/sdk/ts/core/window.ts +++ b/sdk/ts/core/window.ts @@ -30,6 +30,11 @@ export function clockWindow(now: () => number, ttlSec: number): Window { * no two back-to-back signatures share an (keyid, expires) pair — keeping * identical relay requests from colliding in the server's replay store. created * tracks floor(now()), so the pair stays clock-consistent. + * + * ONE INSTANCE PER CLIENT, never one per call. The running maximum is the whole + * mechanism: a window created per request starts from zero, cannot see the + * previous signature, and provides exactly none of the uniqueness it was chosen + * for — while still looking correct at the call site. */ export function monotonicWindow(now: () => number, ttlSec: number): Window { let lastExpires = 0; diff --git a/sdk/ts/resolvers/errors.ts b/sdk/ts/resolvers/errors.ts index d11a9af8..26c324d1 100644 --- a/sdk/ts/resolvers/errors.ts +++ b/sdk/ts/resolvers/errors.ts @@ -87,6 +87,51 @@ export class EndpointRefused extends ResolverError { } } +/** The deployment's allow overlay excluded this Exchange domain, before anything + * was dialled — the class face of Go `ErrExchangeNotPermitted` / Python + * `ExchangeNotPermittedError`. It says nothing about whether the Exchange exists + * or answers, only that this deployment declined to ask, so the remedy is a + * configuration change rather than a retry. */ +export class ExchangeNotPermitted extends ResolverError { + constructor(message: string, options?: { cause?: unknown }) { + super(message, options); + this.name = "ExchangeNotPermitted"; + } +} + +/** The document served at the domain's well-known path describes some other role + * — the class face of Go `ErrManifestNotExchange` / Python + * `ManifestNotExchangeError`. Registration requirements are an Exchange's to + * publish, so a manifest claiming to be an agent, a broker or a publisher is + * refused rather than read for members it has no business carrying. A manifest + * naming no role at all is refused the same way: the field is required by the + * contract, and reading silence as assent would make the check advisory. */ +export class ManifestNotExchange extends ResolverError { + constructor(message: string, options?: { cause?: unknown }) { + super(message, options); + this.name = "ManifestNotExchange"; + } +} + +/** The document arrived and this reader cannot use it — the class face of Go + * `ErrManifestUnusable` / Python `ManifestUnusableError`. A VERDICT and not a + * failed read: the bytes were served, and the next attempt gets the same ones, so + * a caller told to retry retries forever. + * + * The SDK's own requirements reader never throws it. The two ways a manifest can + * disappoint that reader are both deliberate non-errors: a member carrying a type + * the contract does not admit reads as ABSENT, because the projection is shared + * with the endpoint and key faces, and a document that does not parse at all is a + * transport failure. This class exists because the reader seam is injectable and + * one stricter than the SDK's own — validating the whole document, or refusing a + * version — has to be able to say its refusal is final. */ +export class ManifestUnusable extends ResolverError { + constructor(message: string, options?: { cause?: unknown }) { + super(message, options); + this.name = "ManifestUnusable"; + } +} + /** A /.well-known/ramp.json was fetched and parsed but carries a * WellKnownManifest.ver this resolver does not accept: an unrecognised major * version, a value that is not MAJOR.MINOR, or no version at all. The rule is diff --git a/sdk/ts/resolvers/http.ts b/sdk/ts/resolvers/http.ts index ce73f1bd..caf96a0e 100644 --- a/sdk/ts/resolvers/http.ts +++ b/sdk/ts/resolvers/http.ts @@ -265,12 +265,21 @@ export function guardedFetchFromEnv(): FetchLike { return (url) => requestBounded(url, dispatcher, schemeGuardAllows); } -/** Default transport for a resolver whose URL is caller-configured (well-known - * JWKS / ramp.json): a plain fetch. It is NOT SSRF-guarded — the operator, not an - * attacker, chooses that URL, and an on-prem JWKS may legitimately be private. - * Only the WBA resolver (whose host comes from the request-supplied - * Signature-Agent, fetched pre-auth) defaults to `guardedFetch`, matching the Go - * oracle, which guards only its WBA client. */ +/** Default transport for a resolver whose URL is a FIXED, operator-chosen address + * (the well-known JWKS): a plain fetch, NOT SSRF-guarded. An on-prem JWKS may + * legitimately be private, and the operator rather than an attacker chose it. + * + * Which default a resolver takes follows its URL's PROVENANCE, and that is the + * whole rule — the Go oracle states it in the options struct these ports mirror. + * A fixed operator-chosen URL takes this transport. A REQUEST-DERIVED host — the + * WBA directory named by a Signature-Agent header, an Exchange domain read off an + * offer or a registration — takes `guardedFetchFromEnv`, because the party + * choosing the address is not the party running the process. + * + * The endpoint resolver in this package is request-derived and still defaults + * here, which is a known gap being closed separately; it is not the rule. Do not + * reach for this transport for a new resolver without first asking where its URL + * comes from. */ export const defaultFetch: FetchLike = async (url) => { const r = await fetch(url); // Bound the body read even on the unguarded path: a misconfigured / hostile diff --git a/sdk/ts/resolvers/index.ts b/sdk/ts/resolvers/index.ts index b7c58eb1..ba157fab 100644 --- a/sdk/ts/resolvers/index.ts +++ b/sdk/ts/resolvers/index.ts @@ -9,8 +9,11 @@ export { DirectoryUnavailable, EndpointRefused, + ExchangeNotPermitted, KeyExpired, KeyRevoked, + ManifestNotExchange, + ManifestUnusable, ManifestVersionRefused, NoEndpoint, ResolverError, @@ -27,6 +30,12 @@ export { createWBAOfferDirectoryFetch, type OfferDirectoryFetch, } from "./offer-key-cache.ts"; +export { + createWellKnownRequirementsReader, + type RegistrationRequirements, + type WellKnownRequirementsOptions, + type WellKnownRequirementsReader, +} from "./registration-requirements.ts"; export { allowedScheme, blockedAddress } from "./ssrf.ts"; export { createStaticKeyResolver, type StaticKeyResolver } from "./static.ts"; export { diff --git a/sdk/ts/resolvers/registration-requirements.ts b/sdk/ts/resolvers/registration-requirements.ts new file mode 100644 index 00000000..a009382b --- /dev/null +++ b/sdk/ts/resolvers/registration-requirements.ts @@ -0,0 +1,250 @@ +// Reading what one Exchange asks of a registration — the terms revision that +// submitting one accepts, and the schema its registration_data must match — out of +// that Exchange's own /.well-known/ramp.json. TS port of the Go oracle +// (resolvers/registrationrequirements.go). +// +// EVERY READ IS A FRESH FETCH, and that is the whole design. The protocol requires +// it in as many words: a registering client MUST read the terms digest from a +// freshly fetched manifest rather than a cached copy. A cached ENDPOINT is fine — a +// wrong one fails loudly — but a cached DIGEST is not, because a client cannot +// detect staleness locally, so a warm cache would make it echo a value the Exchange +// has already stopped accepting and retry the same refusal until the cache expired. +// +// That is why this is a SEPARATE face rather than a third method on the endpoint +// resolver. That resolver is built out of exactly the mechanism this value may not +// touch — a per-host TTL cache with single-flight coalescing on top — and it exposes +// no bypass. A face that holds no document cache leaves no cache slot to reuse, +// which is what makes the rule structural rather than a convention callers remember. +// +// Not even the compiled validator is held. Memoising it is a cache, and the useful +// key for one is a property of a deployment's threat model rather than of the +// protocol; an application that wants it wraps this face. + +import { isBareDomain } from "../src/hosts.ts"; +import { invalidHost } from "../src/host-ref.ts"; +import { + compileRegistrationSchema, + type RegistrationSchema, + type SchemaVerdict, +} from "../src/regschema.ts"; +import { DirectoryUnavailable, ExchangeNotPermitted, ManifestNotExchange } from "./errors.ts"; +import { type FetchLike, fetchStrict, guardedFetchFromEnv } from "./http.ts"; + +/** What one Exchange asks of a registration. Both members are optional in the + * contract, and their absence is a normal answer rather than a failure. */ +export interface RegistrationRequirements { + /** The manifest's `terms_digest`, `undefined` when the Exchange publishes none. + * Copy it onto `RegisterRequest.terms_digest` unchanged: the request signature + * covers the echo, and that echo is the durable record of which terms revision + * the operator accepted. */ + termsDigest: string | undefined; + /** Validates `registration_data` before anything is signed. `null` in TWO cases + * — the Exchange publishes none, and it publishes one this SDK refuses — and + * `verdict` is what tells them apart. Both are deliberately the same VALUE, + * because the contract requires a client that cannot check locally to send + * anyway: a local check that cannot run must not become a local veto. */ + schema: RegistrationSchema | null; + /** The SDK's answer for the published schema. `not_published` is the ordinary + * absent case; `accepted` means `schema` is usable; anything else names why a + * published schema was refused, which is worth logging and is never worth + * refusing the registration over. */ + verdict: SchemaVerdict; +} + +/** The well-known registration-requirements face, named for the concrete reader + * the oracle exposes (`resolvers.WellKnownRequirementsReader`). */ +export interface WellKnownRequirementsReader { + resolveRegistrationRequirements(exchange: string): Promise; +} + +/** Options for the reader. `ttlMs` and `now` are deliberately absent: it caches + * nothing, so it has no freshness to compute. */ +export interface WellKnownRequirementsOptions { + /** The transport. Omitted, it is the SSRF-GUARDED one: the domain is + * caller-named — an agent registers at whichever Exchange it means to transact + * with, and that domain routinely arrives at runtime rather than from + * configuration — so this is request-derived provenance, which takes the guarded + * default. A deployment that must reach a private or loopback Exchange injects + * its own transport here, or opts out through the SKIP_SSRF / ALLOW_INSECURE + * environment flags. */ + fetch?: FetchLike; + /** Trust allowlist consulted BEFORE the fetch. A domain it rejects never + * reaches the network. */ + allow?: (domain: string) => boolean; + /** The URL scheme used to build `{scheme}://{domain}/.well-known/ramp.json` + * (tests inject "http"). */ + scheme?: string; +} + +/** Read registration requirements from an Exchange's own well-known manifest. */ +export function createWellKnownRequirementsReader( + opts: WellKnownRequirementsOptions = {}, +): WellKnownRequirementsReader { + // Built ONCE per reader, never per read: guardedFetchFromEnv constructs a + // dispatcher, so a per-read default would trade an unguarded dial for a socket + // leak. + const fetchFn: FetchLike = opts.fetch ?? guardedFetchFromEnv(); + const scheme = opts.scheme && opts.scheme !== "" ? opts.scheme : "https"; + const allow = opts.allow; + + return { + async resolveRegistrationRequirements(exchange: string): Promise { + // The SHAPE predicate is the contract's isBareDomain, not the routing tier's + // isBareHost. The two are kept apart deliberately and this leg is where the + // difference bites: nothing upstream has run the contract's rule, and the URL + // below is built by concatenation, so a value carrying a path or userinfo + // would choose WHAT is fetched rather than merely where from. + if (!isBareDomain(exchange)) { + throw invalidHost(exchange, "not a bare domain"); + } + if (allow && !allow(exchange)) { + throw new ExchangeNotPermitted(`exchange ${exchange} not permitted by policy`); + } + const url = `${scheme}://${exchange}/.well-known/ramp.json`; + const body = await fetchStrict(fetchFn, url); + let doc: unknown; + try { + doc = JSON.parse(body); + } catch (err) { + throw new DirectoryUnavailable(`manifest decode ${url}`, { cause: err }); + } + if (doc === null || typeof doc !== "object" || Array.isArray(doc)) { + throw new DirectoryUnavailable(`manifest at ${url} is not a JSON object`); + } + if (!describesExchange(doc)) { + throw new ManifestNotExchange(`host=${exchange}`); + } + const digest = (doc as { terms_digest?: unknown }).terms_digest; + const out: RegistrationRequirements = { + termsDigest: typeof digest === "string" ? digest : undefined, + schema: null, + verdict: "not_published", + }; + // Compiled from the bytes AS SERVED. Every cap the registration-schema rules + // state is defined over those bytes, so the member is sliced out of the body + // rather than re-serialised from the parsed value: JSON.parse discards + // whitespace and reformats numbers, and a document that minifies under the + // cap while exceeding it on the wire would be refused by an Exchange and + // accepted here — the two-privately-chosen-limits failure the contract's + // numbers exist to prevent. + const raw = rawMember(body, "account_registration", "data_schema"); + if (raw !== undefined) { + const compiled = compileRegistrationSchema(raw); + out.schema = compiled.schema; + out.verdict = compiled.verdict; + } + return out; + }, + }; +} + +/** proto-JSON renders an enum as its name or its number, so both are accepted. An + * absent or unrecognised role is not an Exchange. */ +function describesExchange(doc: unknown): boolean { + const role = (doc as { role?: unknown }).role; + return role === "ROLE_EXCHANGE" || role === 2; +} + +/** rawMember returns the EXACT served text of a nested object member, or + * `undefined` when any step of the path is absent. + * + * It exists because JavaScript has no equivalent of a raw-message slice: the only + * way to hand a nested member to a rule defined over the bytes AS SERVED is to + * find its extent in the body. `body` has already parsed as JSON when this runs, + * so the scan below is over known-well-formed input and needs no error path — it + * returns `undefined` for anything it cannot walk, and the caller reads that as + * "no schema published", which is the safe direction. + */ +function rawMember(body: string, ...path: string[]): string | undefined { + let start = 0; + for (let step = 0; step < path.length; step++) { + const found = memberExtent(body, start, path[step] as string); + if (found === undefined) return undefined; + if (step === path.length - 1) return body.slice(found[0], found[1]); + start = found[0]; + } + return undefined; +} + +/** memberExtent finds `key` in the object beginning at `from` and returns the + * [start, end) of its VALUE. */ +function memberExtent(body: string, from: number, key: string): [number, number] | undefined { + let i = skipWhitespace(body, from); + if (body[i] !== "{") return undefined; + i = skipWhitespace(body, i + 1); + if (body[i] === "}") return undefined; + for (;;) { + if (body[i] !== '"') return undefined; + const keyEnd = skipString(body, i); + const name = JSON.parse(body.slice(i, keyEnd)) as string; + i = skipWhitespace(body, keyEnd); + if (body[i] !== ":") return undefined; + const valueStart = skipWhitespace(body, i + 1); + const valueEnd = skipValue(body, valueStart); + if (valueEnd === undefined) return undefined; + if (name === key) return [valueStart, valueEnd]; + i = skipWhitespace(body, valueEnd); + if (body[i] === ",") { + i = skipWhitespace(body, i + 1); + continue; + } + return undefined; + } +} + +/** RFC 8259 whitespace, and nothing wider — the same four bytes the blank-schema + * rule counts. */ +function skipWhitespace(body: string, i: number): number { + while (i < body.length) { + const c = body[i]; + if (c !== " " && c !== "\t" && c !== "\r" && c !== "\n") return i; + i++; + } + return i; +} + +/** skipString returns the index just past a JSON string starting at `i`. */ +function skipString(body: string, i: number): number { + i++; // the opening quote + while (i < body.length) { + const c = body[i]; + if (c === "\\") { + i += 2; + continue; + } + if (c === '"') return i + 1; + i++; + } + return i; +} + +/** skipValue returns the index just past the JSON value starting at `i`, or + * `undefined` when the value does not terminate. */ +function skipValue(body: string, i: number): number | undefined { + const c = body[i]; + if (c === '"') return skipString(body, i); + if (c === "{" || c === "[") { + const close = c === "{" ? "}" : "]"; + let depth = 0; + while (i < body.length) { + const ch = body[i]; + if (ch === '"') { + i = skipString(body, i); + continue; + } + if (ch === "{" || ch === "[") depth++; + else if (ch === "}" || ch === "]") { + depth--; + if (depth === 0) { + return body[i] === close ? i + 1 : undefined; + } + } + i++; + } + return undefined; + } + // A number, or one of the three literals: everything up to the next structural + // character. The body has already parsed, so whatever lies here is well formed. + const end = /[\s,}\]]/.exec(body.slice(i)); + return end === null ? body.length : i + end.index; +} diff --git a/sdk/ts/tests/account.test.ts b/sdk/ts/tests/account.test.ts new file mode 100644 index 00000000..5b8a2fab --- /dev/null +++ b/sdk/ts/tests/account.test.ts @@ -0,0 +1,343 @@ +import { describe, expect, it } from "vitest"; + +import { createClient, RampCallError } from "../client/index.ts"; +import type { UnaryRequest, UnarySend } from "../client/index.ts"; +import { + ExchangeNotPermitted, + ManifestNotExchange, + ManifestUnusable, + type RegistrationRequirements, +} from "../resolvers/index.ts"; +import { compileRegistrationSchema } from "../src/regschema.ts"; +import { retrievalAuthFailureDetail } from "../src/errordetail.ts"; +import { invalidHost } from "../src/host-ref.ts"; + +/** A send that records what it was given and answers a fixed body. */ +function recordingSend( + answer: unknown, + status = 200, +): { send: UnarySend; seen: UnaryRequest[] } { + const seen: UnaryRequest[] = []; + const send: UnarySend = async (req) => { + seen.push(req); + return { status, body: JSON.stringify(answer) }; + }; + return { send, seen }; +} + +function bodyOf(req: UnaryRequest): Record { + return JSON.parse(new TextDecoder().decode(req.body)) as Record; +} + +const SCHEMA = + '{"type":"object","required":["legal_entity"],"properties":{"legal_entity":{"type":"string"}}}'; + +/** Fails two ways at once: a required member is absent, which is a whole-object + * violation the wire reports with an EMPTY path, and a member that IS present breaks + * its pattern, which is reported against its own pointer. One refusal carrying both is + * what a consumer has to render, and the empty path is the half easiest to render + * wrong. */ +const BOTH_SHAPES_SCHEMA = + '{"type":"object","required":["legal_entity"],' + + '"properties":{"vat_id":{"type":"string","pattern":"^[A-Z]{2}[0-9]+$"}}}'; + +/** Drive a pre-check refusal against `schema`, so the tests below share one + * arrangement and differ only in what they read off the failure. */ +async function refusedByTheSchema( + schemaSource: string, + payload: Record, +): Promise { + const { schema, verdict } = compileRegistrationSchema(schemaSource); + expect(verdict).toBe("accepted"); + const { send, seen } = recordingSend({ ver: "1.0", billing_ref: "acct-1" }); + + const err = await client(send, { termsDigest: undefined, schema, verdict }) + .register({ exchange: "exchange.test", registration_data: payload }) + .catch((e: unknown) => e); + + expect(err).toBeInstanceOf(RampCallError); + expect((err as RampCallError).kind).toBe("malformed"); + expect(seen).toHaveLength(0); + return err as RampCallError; +} + +const ENDPOINT = "https://exchange.test"; +const endpointResolver = { resolveEndpoint: async () => ENDPOINT }; + +const publishesNothing: RegistrationRequirements = { + termsDigest: undefined, + schema: null, + verdict: "not_published", +}; + +function client( + send: UnarySend, + reqs: RegistrationRequirements | (() => Promise) = publishesNothing, +) { + return createClient("https://home.invalid", { + endpointResolver, + send, + guardedSend: send, + registrationRequirements: { + resolveRegistrationRequirements: async () => { + if (typeof reqs === "function") return reqs(); + return reqs; + }, + }, + }); +} + +describe("register", () => { + it("addresses the RPC, stamps ver and keeps the caller's recipient", async () => { + const { send, seen } = recordingSend({ ver: "1.0", billing_ref: "acct-1", active: true }); + const resp = await client(send).register({ + exchange: "exchange.test", + registration_data: { legal_entity: "Acme" }, + }); + expect(resp.billing_ref).toBe("acct-1"); + const req = seen[0] as UnaryRequest; + expect(req.url).toBe(`${ENDPOINT}/ramp.v1.ExchangeService/Register`); + const body = bodyOf(req); + expect(body["ver"]).toBe("1.0"); + expect(body["exchange"]).toBe("exchange.test"); + // No idempotency key is minted: the message carries none. + expect(body["idempotency_key"]).toBeUndefined(); + }); + + it("echoes the freshly fetched terms digest", async () => { + const digest = `sha256:${"ab".repeat(32)}`; + const { send, seen } = recordingSend({ ver: "1.0", billing_ref: "acct-1" }); + await client(send, { ...publishesNothing, termsDigest: digest }).register({ + exchange: "exchange.test", + }); + expect(bodyOf(seen[0] as UnaryRequest)["terms_digest"]).toBe(digest); + }); + + it("leaves a caller-supplied digest alone", async () => { + const mine = `sha256:${"cd".repeat(32)}`; + const { send, seen } = recordingSend({ ver: "1.0", billing_ref: "acct-1" }); + // The reader would throw if it were consulted, which is how this asserts that a + // caller managing its own requirements suppresses the read. + await client(send, async () => { + throw new Error("the requirements read must not happen"); + }).register({ exchange: "exchange.test", terms_digest: mine }); + expect(bodyOf(seen[0] as UnaryRequest)["terms_digest"]).toBe(mine); + }); + + it("refuses an out-of-bounds payload before signing", async () => { + const tooMany: Record = {}; + for (let i = 0; i <= 64; i++) tooMany[`m${i}`] = "v"; + const { send, seen } = recordingSend({}); + await expect( + client(send).register({ exchange: "exchange.test", registration_data: tooMany }), + ).rejects.toBeInstanceOf(RampCallError); + expect(seen).toHaveLength(0); + }); + + it("refuses an unaddressed request before signing", async () => { + const { send, seen } = recordingSend({}); + for (const exchange of ["", "https://exchange.test", "exchange.test/path"]) { + await expect(client(send).register({ exchange })).rejects.toBeInstanceOf(RampCallError); + } + expect(seen).toHaveLength(0); + }); + + // The pre-check working: a schema the SDK can use, and a payload that fails it. + // Refused locally, before anything is signed, with the offending member named — the + // point of reading the schema at all is that the caller learns this without a round + // trip. + it("pre-checks the published schema", async () => { + const { schema, verdict } = compileRegistrationSchema(SCHEMA); + expect(verdict).toBe("accepted"); + const { send, seen } = recordingSend({ ver: "1.0", billing_ref: "acct-1" }); + const withSchema: RegistrationRequirements = { + termsDigest: undefined, + schema, + verdict, + }; + + const err = await client(send, withSchema) + .register({ exchange: "exchange.test", registration_data: { trading_name: "Acme" } }) + .catch((e: unknown) => e); + expect(err).toBeInstanceOf(RampCallError); + expect((err as RampCallError).kind).toBe("malformed"); + expect((err as RampCallError).message).toContain("legal_entity"); + expect(seen).toHaveLength(0); + + // The conforming payload goes through against the same schema. + await client(send, withSchema).register({ + exchange: "exchange.test", + registration_data: { legal_entity: "Acme" }, + }); + expect(seen).toHaveLength(1); + }); + + // What the pre-check computed is what the caller gets. An Exchange attaches this same + // list, from this same validator, when it refuses the same payload — so a consumer that + // renders one refusal renders both, and nothing has to parse the members back out of a + // sentence. + it("carries the field errors the pre-check found", async () => { + const err = await refusedByTheSchema(BOTH_SHAPES_SCHEMA, { vat_id: "de1" }); + + expect(err.detail?.registration_failure?.reason).toBe( + "REGISTRATION_FAILURE_REASON_INVALID_REGISTRATION_DATA", + ); + // The PATHS are what this asserts, never the constraint prose beside them: that text + // comes from each language's validator library, and the contract calls it + // non-authoritative for exactly that reason. + const paths = (err.detail?.registration_failure?.field_errors ?? []).map( + (f) => f.path ?? "", + ); + expect(paths).toEqual(["", "/vat_id"]); + }); + + // An empty path addresses the whole object, which is how a missing required member is + // reported. Rendering a bare ": ..." there would read as a member with no name. + it("renders a whole-object violation with no member name", async () => { + const err = await refusedByTheSchema(BOTH_SHAPES_SCHEMA, { vat_id: "de1" }); + + expect(err.message).not.toContain("publishes: : "); + // The member that HAS a name still carries it, so the fix did not drop the separator + // everywhere. + expect(err.message).toContain("/vat_id: "); + }); + + // A schema the SDK REFUSES is skipped, never a veto. The contract is explicit that + // refusing locally and declining to send would turn a rule about reading a third + // party's document into a denial of service against the caller's own user, so the + // Exchange's own enforcement stays the deciding check. + it("does not let an unusable schema block the send", async () => { + const { schema, verdict } = compileRegistrationSchema( + '{"$schema":"https://json-schema.org/draft/2019-09/schema","type":"object"}', + ); + expect(schema).toBeNull(); + expect(verdict).toBe("wrong_dialect"); + const { send, seen } = recordingSend({ ver: "1.0", billing_ref: "acct-1" }); + + await client(send, { termsDigest: undefined, schema, verdict }).register({ + exchange: "exchange.test", + registration_data: { anything: "goes" }, + }); + expect(seen).toHaveLength(1); + }); + + // A failed READ is classified the way the routing tier classifies its own: a value + // this deployment or the Exchange refused is FINAL, anything else is a transport + // failure worth retrying. Without the split a caller retries a refusal forever. + it("classifies a refused requirements read as final and a transport failure as retryable", async () => { + for (const [thrown, kind] of [ + [new ExchangeNotPermitted("blocked"), "not_sent"], + [new ManifestNotExchange("host=exchange.test"), "not_sent"], + // The SDK's own reader never throws this one: it reads an off-spec member as + // absent and treats an undecodable document as a transport failure. A stricter + // INJECTED reader throws it, and the document arrives unusable again next time. + [new ManifestUnusable('ver "2.0" has major 2, accept major 1'), "not_sent"], + // Reachable only through an INJECTED reader with its own host rule — the verb's + // own recipient check runs this one first. Retryable would be wrong: a value that + // is not a host will not become one on a later attempt. + [invalidHost("exchange.test/path", "not a bare domain"), "not_sent"], + [new Error("connection reset"), "unreachable"], + ] as const) { + const { send, seen } = recordingSend({}); + const err = await client(send, () => Promise.reject(thrown)) + .register({ exchange: "exchange.test" }) + .catch((e: unknown) => e); + expect(err).toBeInstanceOf(RampCallError); + expect((err as RampCallError).kind).toBe(kind); + expect(seen).toHaveLength(0); + } + }); +}); + +describe("getAccountStatus", () => { + it("addresses the RPC and reads an accountless answer as a normal one", async () => { + const { send, seen } = recordingSend({ ver: "1.0", billing_ref: "", active: false }); + const resp = await client(send).getAccountStatus({ exchange: "exchange.test" }); + expect(resp.billing_ref).toBe(""); + expect(resp.active).toBe(false); + expect((seen[0] as UnaryRequest).url).toBe( + `${ENDPOINT}/ramp.v1.ExchangeService/GetAccountStatus`, + ); + expect(bodyOf(seen[0] as UnaryRequest)["ver"]).toBe("1.0"); + }); + + it("refuses an unaddressed request before signing", async () => { + const { send, seen } = recordingSend({}); + await expect(client(send).getAccountStatus({ exchange: "" })).rejects.toBeInstanceOf( + RampCallError, + ); + expect(seen).toHaveLength(0); + }); +}); + +describe("the peer's own sentence", () => { + it("comes back as a value when the peer sent a typed reason", async () => { + const { send } = recordingSend( + { + code: "failed_precondition", + message: "envelope prose", + details: [ + { + type: "ramp.v1.ErrorDetail", + debug: { + domain: "ramp.v1.ExchangeService", + message: "terms have moved", + registrationFailure: { reason: "REGISTRATION_FAILURE_REASON_TERMS_DIGEST_STALE" }, + }, + }, + ], + }, + 412, + ); + const err = await client(send) + .register({ exchange: "exchange.test" }) + .catch((e: unknown) => e); + expect(err).toBeInstanceOf(RampCallError); + expect((err as RampCallError).peerMessage).toBe("terms have moved"); + // The token stays a token: prose never leaks into the field a caller branches on. + expect((err as RampCallError).reason).not.toContain(" "); + }); + + // The content leg builds a typed detail LOCALLY, out of the edge's refusal token: + // the token is the edge's, the sentence around it is this SDK's. So a detail is + // present and the peer's message is still empty — the one case that separates + // "fill it from the detail" from "fill it where the peer's own answer was decoded". + it("stays empty for a detail this SDK synthesized", () => { + const err = new RampCallError({ + kind: "refused", + op: "fetch content", + status: 403, + reason: "keyid_mismatch", + detail: retrievalAuthFailureDetail( + "ramp.v1.Edge", + "delivery refused: keyid_mismatch", + "RETRIEVAL_AUTH_FAILURE_REASON_KEYID_MISMATCH", + ), + }); + // The typed reason still reaches the caller — this is not about losing it. + expect(err.detail).toBeDefined(); + expect(err.peerMessage).toBe(""); + }); + + // The registration pre-check is the same rule with no peer involved at all: the + // request never left the process, so there is no remote party whose words this field + // could hold. A detail is present because the schema failures are worth carrying; the + // sentence around them is ours. + it("stays empty for a detail the registration pre-check synthesized", async () => { + const err = await refusedByTheSchema(SCHEMA, { trading_name: "Acme" }); + + expect(err.detail).toBeDefined(); + expect(err.peerMessage).toBe(""); + }); + + it("is empty when the answer carried no typed reason", async () => { + const { send } = recordingSend({ code: "unavailable", message: "draining" }, 503); + const err = await client(send) + .getAccountStatus({ exchange: "exchange.test" }) + .catch((e: unknown) => e); + expect((err as RampCallError).peerMessage).toBe(""); + // The transport's own account is not lost, only kept out of the field that + // claims to hold the peer's words. + expect((err as RampCallError).message).toContain("draining"); + }); +}); diff --git a/sdk/ts/tests/client-request.parity.test.ts b/sdk/ts/tests/client-request.parity.test.ts index 0f57ae0f..0973785c 100644 --- a/sdk/ts/tests/client-request.parity.test.ts +++ b/sdk/ts/tests/client-request.parity.test.ts @@ -97,6 +97,17 @@ async function call(name: string): Promise { const keys = await agentKeys(); const options = { requester: REQUESTER, + // Register reads an Exchange's published requirements before it signs. What + // this corpus records is the path and the envelope, so the read is stubbed to + // an Exchange that publishes neither a digest nor a schema — the ordinary + // pass-through case. Inert for every other verb. + registrationRequirements: { + resolveRegistrationRequirements: async () => ({ + termsDigest: undefined, + schema: null, + verdict: "not_published" as const, + }), + }, signer: { privKey: keys.privateKey, keyid: "agent.v1" }, agentPublicKey: keys.publicKey, endpointResolver, @@ -125,6 +136,12 @@ async function call(name: string): Promise { ); } else if (name.startsWith("execute")) { await client.execute(await verifiedOffer(), { idempotencyKey: PINNED }); + } else if (name.startsWith("register")) { + const req: Record = { exchange: ISSUER }; + if (name === "register_caller_ver_wins") req["ver"] = "9.9"; + await client.register(req); + } else if (name === "get_account_status") { + await client.getAccountStatus({ exchange: ISSUER }); } else if (name === "resolve") { await createBrokerClient("https://broker.test", options).resolve({}); } else if (name.startsWith("push_resources")) { diff --git a/sdk/ts/tests/client-requirements-single-reader.guard.test.ts b/sdk/ts/tests/client-requirements-single-reader.guard.test.ts new file mode 100644 index 00000000..ae1ca528 --- /dev/null +++ b/sdk/ts/tests/client-requirements-single-reader.guard.test.ts @@ -0,0 +1,71 @@ +import { readFileSync } from "node:fs"; +import { fileURLToPath } from "node:url"; +import { describe, expect, it } from "vitest"; + +// The default requirements reader is built ONCE per client, never once per call. +// +// The reader's default transport is the SSRF-guarded one, and guardedFetchFromEnv() +// constructs a dispatcher on every invocation. So `?? createWellKnownRequirementsReader()` +// written at the call site opens one connection pool per registration and closes none — +// the guard fix and this one are the same change, and splitting them would trade an +// unguarded dial for a socket leak. +// +// It is resolved in resolve(), beside the verifier and the two sends, which is the tier +// Go resolves the same default at (connect.NewClient) and where every other per-client +// dependency already lives. +// +// Structural rather than behavioral, per the same exception resolvers-wba-path-dedup +// takes: what went wrong was WHERE the constructor was called, and a leaked dispatcher +// has no observable it reports. Counting sockets would pin Node's pooling rather than +// this decision. The detector is whitespace-tolerant so a formatter reflow cannot let a +// revert slip; the meta-tests below exercise that. + +const clientPath = fileURLToPath(new URL("../client/index.ts", import.meta.url)); + +function readClient(): string { + return readFileSync(clientPath, "utf8"); +} + +/** Every construction of the default reader, wherever it appears. */ +const CONSTRUCTS = /createWellKnownRequirementsReader\s*\(/g; + +/** The one construction that is allowed: the field on the resolved-once record. */ +const IN_RESOLVED = + /requirements\s*:\s*\n?\s*opts\.registrationRequirements\s*\?\?\s*\n?\s*createWellKnownRequirementsReader\s*\(\s*\)/; + +/** The shape this replaced, and the one a revert would reintroduce: the reader built + * inside the verb's own helper, where it runs per registration. */ +const AT_THE_CALL_SITE = + /const\s+reader\s*=\s*\n?\s*r?\.?opts\.registrationRequirements\s*\?\?\s*\n?\s*createWellKnownRequirementsReader\s*\(\s*\)/; + +describe("the default requirements reader is constructed once per client", () => { + it("is constructed in exactly one place", () => { + expect(readClient().match(CONSTRUCTS)).toHaveLength(1); + }); + + it("constructs it on the resolved-once record, with the transports", () => { + expect(IN_RESOLVED.test(readClient())).toBe(true); + }); + + it("does not construct it at the call site", () => { + expect(AT_THE_CALL_SITE.test(readClient())).toBe(false); + }); + + // Meta-tests: the detectors match the shapes they claim to, reflowed as a + // formatter would leave them. Without these a typo in a regex reads as a pass. + it("detects the call-site shape it exists to forbid", () => { + expect( + AT_THE_CALL_SITE.test( + "const reader =\n\t\tr.opts.registrationRequirements ?? createWellKnownRequirementsReader();", + ), + ).toBe(true); + }); + + it("detects the resolved-once shape across a line break", () => { + expect( + IN_RESOLVED.test( + "requirements:\n\t\t\topts.registrationRequirements ?? createWellKnownRequirementsReader(),", + ), + ).toBe(true); + }); +}); diff --git a/sdk/ts/tests/registration-requirements.test.ts b/sdk/ts/tests/registration-requirements.test.ts new file mode 100644 index 00000000..f626cf45 --- /dev/null +++ b/sdk/ts/tests/registration-requirements.test.ts @@ -0,0 +1,122 @@ +import { describe, expect, it } from "vitest"; + +import { + createWellKnownRequirementsReader, + ExchangeNotPermitted, + ManifestNotExchange, +} from "../resolvers/index.ts"; + +const SCHEMA = `{"type":"object","required":["legal_entity"],"properties":{"legal_entity":{"type":"string"}}}`; + +/** serve builds a fetch that answers one manifest body and counts calls. */ +function serve(body: string): { fetch: typeof fetch; hits: () => number } { + let hits = 0; + const fetchFn = (async () => { + hits++; + return new Response(body, { status: 200, headers: { "content-type": "application/json" } }); + }) as unknown as typeof fetch; + return { fetch: fetchFn, hits: () => hits }; +} + +function manifest(extra: Record = {}, role: unknown = "ROLE_EXCHANGE"): string { + return JSON.stringify({ role, endpoint: "https://exchange.test", ...extra }); +} + +const reader = (body: string, allow?: (d: string) => boolean) => { + const s = serve(body); + return { + r: createWellKnownRequirementsReader({ + fetch: s.fetch, + scheme: "http", + ...(allow !== undefined ? { allow } : {}), + }), + hits: s.hits, + }; +}; + +describe("the registration-requirements reader", () => { + it("reads the digest and compiles the published schema", async () => { + const digest = `sha256:${"ab".repeat(32)}`; + const { r } = reader( + manifest({ terms_digest: digest, account_registration: { data_schema: JSON.parse(SCHEMA) } }), + ); + const got = await r.resolveRegistrationRequirements("exchange.test"); + expect(got.termsDigest).toBe(digest); + expect(got.verdict).toBe("accepted"); + // The schema is the one that was served, not merely present. + expect(got.schema?.validate({}).length).toBeGreaterThan(0); + expect(got.schema?.validate({ legal_entity: "Acme" })).toEqual([]); + }); + + it("fetches afresh on every read", async () => { + const { r, hits } = reader(manifest({ terms_digest: `sha256:${"ab".repeat(32)}` })); + await r.resolveRegistrationRequirements("exchange.test"); + await r.resolveRegistrationRequirements("exchange.test"); + expect(hits()).toBe(2); + }); + + it("treats publishing neither member as a normal answer", async () => { + const { r } = reader(manifest()); + const got = await r.resolveRegistrationRequirements("exchange.test"); + expect(got.termsDigest).toBeUndefined(); + expect(got.schema).toBeNull(); + expect(got.verdict).toBe("not_published"); + }); + + it("reports a refused schema as a verdict, never as a throw", async () => { + const { r } = reader( + manifest({ + account_registration: { + data_schema: { $schema: "https://json-schema.org/draft/2019-09/schema", type: "object" }, + }, + }), + ); + const got = await r.resolveRegistrationRequirements("exchange.test"); + expect(got.verdict).toBe("wrong_dialect"); + expect(got.schema).toBeNull(); + }); + + // The cap is defined over the bytes AS SERVED. A schema padded past the cap on + // the wire that would minify under it must be refused here exactly as the + // Exchange refuses it, or a client pre-checks against a schema nobody enforces. + it("measures the schema over the served bytes, not a re-encoding", async () => { + const padded = `{"type":"object",${" ".repeat(20_000)}"title":"x"}`; + const body = `{"role":"ROLE_EXCHANGE","account_registration":{"data_schema":${padded}}}`; + const { r } = reader(body); + const got = await r.resolveRegistrationRequirements("exchange.test"); + expect(got.verdict).toBe("too_large"); + }); + + it("refuses before dialling", async () => { + const bad = reader(manifest()); + await expect(bad.r.resolveRegistrationRequirements("exchange.test/path")).rejects.toThrow(); + expect(bad.hits()).toBe(0); + + const blocked = reader(manifest(), () => false); + await expect(blocked.r.resolveRegistrationRequirements("exchange.test")).rejects.toBeInstanceOf( + ExchangeNotPermitted, + ); + expect(blocked.hits()).toBe(0); + }); + + it("refuses a manifest that is not an Exchange", async () => { + for (const body of [ + manifest({}, "ROLE_BROKER"), + manifest({}, "ROLE_AGENT"), + manifest({}, "ROLE_PUBLISHER"), + // No role at all. The contract makes the field required, so reading silence + // as assent would leave the check advisory. + `{"endpoint":"https://exchange.test"}`, + ]) { + const { r } = reader(body); + await expect(r.resolveRegistrationRequirements("exchange.test")).rejects.toBeInstanceOf( + ManifestNotExchange, + ); + } + }); + + it("accepts the role as a proto-JSON number", async () => { + const { r } = reader(manifest({}, 2)); + await expect(r.resolveRegistrationRequirements("exchange.test")).resolves.toBeDefined(); + }); +}); diff --git a/sdk/ts/tests/resolvers-transport-defaults.guard.test.ts b/sdk/ts/tests/resolvers-transport-defaults.guard.test.ts new file mode 100644 index 00000000..f5e3de05 --- /dev/null +++ b/sdk/ts/tests/resolvers-transport-defaults.guard.test.ts @@ -0,0 +1,153 @@ +import { createServer, type Server } from "node:http"; +import type { AddressInfo } from "node:net"; + +import { afterEach, beforeEach, describe, expect, it } from "vitest"; + +import { + createWellKnownEndpointResolver, + createWellKnownKeyResolver, + createWellKnownRequirementsReader, +} from "../resolvers/index.ts"; + +// Which transport each resolver takes when the caller injects none. +// +// # Why this is a test and not a sentence +// +// The rule is that a resolver's default follows its URL's PROVENANCE: a fixed, +// operator-chosen address takes the plain transport, a host another party named +// takes the guarded one. That rule is written down in three places — the Go +// oracle's WellKnownOptions.HTTP comment, docs/design-history.md, and the +// published threat model, which states it per language in a table. +// +// It has now drifted from the code twice. The scheme guard was added to the dial +// and not walked back through its callers, so a doc sentence beginning "every +// fetch" stayed standing after it stopped being true; and these ports mirrored +// the oracle's options struct without mirroring the rule, then explained the +// result in a docstring that had also stopped being true. Prose cannot detect +// either. A socket can. +// +// SO: CHANGING ANY ROW BELOW OBLIGES AN EDIT TO THE PUBLISHED THREAT MODEL +// (website/src/content/docs/security/threat-model.mdx), which names these +// postures per language. That is the whole point of the gate — the failure is +// the reminder. +// +// # One row records a gap, not a decision +// +// `endpoint resolver` REACHES here and REFUSES in Go. That is not a considered +// divergence: the host is an Offer.exchange domain, so by the rule above it +// belongs on the guarded transport in every language, and this port has not +// caught up. It is tracked separately, and the threat model names it. Do not +// read this row as endorsement — read it as the thing that will fail, loudly and +// on purpose, in the change that fixes it. +// +// # What is not here +// +// The WBA directory resolver's guarded default is pinned already, by +// resolvers-ssrf.test.ts and its Go and Python counterparts; its guard is +// additionally not opt-outable, which those cover and this would not. Asserting +// it twice would be one behaviour with two owners. +// +// This is deliberately NOT a shared corpus. A *-vectors.json has to be replayed +// by all three languages against the same expectations, and the expectations +// here legitimately DIFFER per language — that difference is the subject. + +const TOUCHED = ["SKIP_SSRF", "ALLOW_INSECURE", "HTTP_PROXY", "HTTPS_PROXY"]; +let saved: Record; + +// Both guards are env-driven, so the flags are cleared rather than inherited from +// whatever shell ran the suite: without this a developer with SKIP_SSRF set sees +// the guarded rows fail for a reason that has nothing to do with the defaults. +beforeEach(() => { + saved = {}; + for (const k of TOUCHED) { + saved[k] = process.env[k]; + delete process.env[k]; + } +}); + +afterEach(() => { + for (const k of TOUCHED) { + if (saved[k] === undefined) delete process.env[k]; + else process.env[k] = saved[k]; + } +}); + +/** Whether a resolver reached the loopback origin. A guarded default refuses the + * reserved address before the request leaves the process, so "the handler ran" is + * the observable that separates the two transports. */ +async function dialObserved( + exercise: (host: string) => Promise, +): Promise { + let reached = false; + const server: Server = createServer((req, res) => { + reached = true; + res.writeHead(200, { "content-type": "application/json" }); + // Anchored to the host that served it, so the endpoint face's own same-host + // rule cannot be what refuses. + res.end( + JSON.stringify({ + role: "ROLE_EXCHANGE", + endpoint: `https://${req.headers.host}`, + keys: [], + }), + ); + }); + await new Promise((resolve) => server.listen(0, "127.0.0.1", resolve)); + const { port } = server.address() as AddressInfo; + try { + // The rejection is deliberately swallowed: a guarded refusal and a happy + // answer are both expected outcomes, and `reached` is what says which. + await exercise(`127.0.0.1:${port}`).catch(() => undefined); + } finally { + await new Promise((resolve) => server.close(() => resolve())); + } + return reached; +} + +// `reaches: true` means the default is the PLAIN transport. +const ROWS: { + name: string; + reaches: boolean; + why: string; + exercise: (host: string) => Promise; +}[] = [ + { + name: "key resolver dials plain", + reaches: true, + why: + "its URL is a fixed, operator-chosen JWKS address — nobody else can point it, " + + "and an on-prem directory may legitimately be private", + exercise: (host) => + createWellKnownKeyResolver(`http://${host}/.well-known/ramp.json`, {}).resolve("ex.v1"), + }, + { + name: "endpoint resolver dials plain — a tracked gap, not a decision", + reaches: true, + why: + "its host is an Offer.exchange domain, so another party chose the address and " + + "the rule says guarded. Go already refuses here; this port has not caught up", + exercise: (host) => + createWellKnownEndpointResolver({ scheme: "http" }).resolveEndpoint(host), + }, + { + name: "requirements reader dials guarded", + reaches: false, + why: "its host is a RegisterRequest.exchange domain, named by the caller per call", + exercise: (host) => + createWellKnownRequirementsReader({ scheme: "http" }).resolveRegistrationRequirements(host), + }, +]; + +describe("resolver transport defaults", () => { + for (const row of ROWS) { + it(row.name, async () => { + const got = await dialObserved(row.exercise); + const verb = (v: boolean) => (v ? "reached" : "refused"); + expect( + got, + `default transport ${verb(got)} the loopback origin, want ${verb(row.reaches)} — ${row.why}. ` + + "If this change is intended, the per-language table in the published threat model has to change with it.", + ).toBe(row.reaches); + }); + } +}); diff --git a/sdk/ts/tests/synthesized-detail.parity.test.ts b/sdk/ts/tests/synthesized-detail.parity.test.ts new file mode 100644 index 00000000..cb572a6b --- /dev/null +++ b/sdk/ts/tests/synthesized-detail.parity.test.ts @@ -0,0 +1,164 @@ +// The details this SDK BUILDS rather than receives, replayed against the Go oracle. +// +// Decoding a peer's ErrorDetail is already pinned elsewhere. These two are different: +// nothing on the wire says what they should contain, so the domain, the sentence and the +// typed reason are AUTHORED — three times, in three languages, from three copies of one +// decision. That is the shape that drifts silently, and it had: this port derived each +// edge reason's enum name by uppercasing the token, which is right for two of the eleven +// tokens the protocol records and wrong for the rest. +// +// Field errors are compared by PATH and never by text. The constraint prose comes from a +// different validator library in each language and the contract calls it +// non-authoritative, so pinning it here would pin an accident. +import { createServer } from "node:http"; +import { describe, expect, it } from "vitest"; + +import { createClient, RampCallError } from "../client/index.ts"; +import type { UnaryRequest, UnarySend } from "../client/index.ts"; +import type { RegistrationRequirements } from "../resolvers/index.ts"; +import { REASON_FIELDS, reason } from "../src/errordetail.ts"; +import { compileRegistrationSchema } from "../src/regschema.ts"; +import vectorsFile from "../../go/connect/testdata/synthesized-detail-vectors.json"; + +interface PrecheckVector { + name: string; + schema: string; + data: Record; + domain: string; + message: string; + reason_field: string; + reason_enum: string; + field_error_paths: string[]; +} + +interface EdgeRefusalVector { + name: string; + reason_token: string; + domain: string; + message: string; + reason_field: string; + reason_enum: string; +} + +const { registration_precheck: precheckVectors, edge_refusal: edgeVectors } = + vectorsFile as unknown as { + registration_precheck: PrecheckVector[]; + edge_refusal: EdgeRefusalVector[]; + }; + +const ENDPOINT = "https://exchange.test"; + +function noSend(): UnarySend { + return async (_req: UnaryRequest) => { + throw new Error("the pre-check refusal must never reach the wire"); + }; +} + +/** Assert the detail on `err` matches what the oracle recorded for `v`. */ +function expectsRecordedDetail( + err: RampCallError, + v: { domain: string; message: string; reason_field: string; reason_enum: string }, +): void { + expect(err.detail).toBeDefined(); + const detail = err.detail as NonNullable; + expect(detail.domain ?? "").toBe(v.domain); + expect(detail.message ?? "").toBe(v.message); + // Exactly the recorded oneof member is populated, and only it. + const record = detail as unknown as Record; + expect(REASON_FIELDS.filter((f) => record[f] != null)).toEqual([v.reason_field]); + const got = reason(detail); + expect(got?.field).toBe(v.reason_field); + expect(got?.value).toBe(v.reason_enum); +} + +describe("the registration pre-check's synthesized detail", () => { + for (const v of precheckVectors) { + it(v.name, async () => { + const { schema, verdict } = compileRegistrationSchema(v.schema); + expect(verdict, "the row's own schema does not compile").toBe("accepted"); + const requirements: RegistrationRequirements = { + termsDigest: undefined, + schema, + verdict, + }; + const client = createClient("https://home.invalid", { + endpointResolver: { resolveEndpoint: async () => ENDPOINT }, + send: noSend(), + guardedSend: noSend(), + registrationRequirements: { + resolveRegistrationRequirements: async () => requirements, + }, + }); + + const err = (await client + .register({ exchange: "exchange.test", registration_data: v.data }) + .catch((e: unknown) => e)) as RampCallError; + + expect(err).toBeInstanceOf(RampCallError); + expect(err.kind).toBe("malformed"); + expectsRecordedDetail(err, v); + const paths = (err.detail?.registration_failure?.field_errors ?? []).map( + (f) => f.path ?? "", + ); + expect(paths).toEqual(v.field_error_paths); + }); + } +}); + +describe("the content leg's synthesized detail", () => { + /** One loopback edge that refuses with whatever token the path asks for. A real + * socket, because this port has no seam to inject the content fetch through — the + * guard and the scheme gate both refuse loopback and plaintext by default, which is + * what the two flags are for. */ + async function refusedByTheEdge(token: string): Promise { + const server = createServer((_req, res) => { + res.writeHead(403, { "content-type": "application/json" }); + res.end(JSON.stringify({ error: "denied", reason: token })); + }); + await new Promise((done) => server.listen(0, "127.0.0.1", () => done())); + const port = (server.address() as { port: number }).port; + + const saved = [process.env["SKIP_SSRF"], process.env["ALLOW_INSECURE"]]; + process.env["SKIP_SSRF"] = "true"; + process.env["ALLOW_INSECURE"] = "true"; + try { + const keys = (await crypto.subtle.generateKey({ name: "Ed25519" }, true, [ + "sign", + "verify", + ])) as CryptoKeyPair; + const client = createClient("https://exchange.test", { + signer: { privKey: keys.privateKey, keyid: "agent.v1" }, + agentPublicKey: keys.publicKey, + }); + return (await client + .fetch(`http://127.0.0.1:${port}/doc`) + .catch((e: unknown) => e)) as RampCallError; + } finally { + if (saved[0] === undefined) delete process.env["SKIP_SSRF"]; + else process.env["SKIP_SSRF"] = saved[0]; + if (saved[1] === undefined) delete process.env["ALLOW_INSECURE"]; + else process.env["ALLOW_INSECURE"] = saved[1]; + server.close(); + } + } + + for (const v of edgeVectors) { + it(v.name, async () => { + const err = await refusedByTheEdge(v.reason_token); + + expect(err).toBeInstanceOf(RampCallError); + // The raw token reaches the caller either way; only the typed reason is at + // stake, so a row that records none must still carry the refusal. + expect(err.reasonOf()).toBe(v.reason_token); + + if (v.reason_field === "") { + // The protocol records no reason for this token, or records two and so + // cannot say which check ran. Promoting one would attribute a failure the + // wire never attributed. + expect(err.detail).toBeUndefined(); + return; + } + expectsRecordedDetail(err, v); + }); + } +}); diff --git a/website/src/content/docs/components/agent-sdk/fetch-flow.mdx b/website/src/content/docs/components/agent-sdk/fetch-flow.mdx index 9f7abcdf..58a4e58a 100644 --- a/website/src/content/docs/components/agent-sdk/fetch-flow.mdx +++ b/website/src/content/docs/components/agent-sdk/fetch-flow.mdx @@ -258,7 +258,7 @@ type NoOfferError struct { // TransactionDeniedError -- Exchange denied the transaction. type TransactionDeniedError struct { Reason DenialReason // ACCOUNT_NOT_REGISTERED, ACCOUNT_INACTIVE, DELEGATION_INVALID, etc. - Exchange string // from TransactionDenial.exchange -- where to call Register + Exchange string // from TransactionDenial.exchange -- the domain to pass to the SDK's Register verb OfferID string } diff --git a/website/src/content/docs/components/agent-sdk/overview.mdx b/website/src/content/docs/components/agent-sdk/overview.mdx index d8d25981..f26794e7 100644 --- a/website/src/content/docs/components/agent-sdk/overview.mdx +++ b/website/src/content/docs/components/agent-sdk/overview.mdx @@ -438,6 +438,8 @@ The concrete implementations differ in persistence strategy (in-memory vs Redis) | TypeScript | [`gen/ts/`](https://github.com/RAMP-Protocol/protocol/tree/main/gen/ts/) (types) + [`sdk/ts/`](https://github.com/RAMP-Protocol/protocol/tree/main/sdk/ts/) (hand-written SDK: `src`, `core`, `resolvers`, `client`, `hono`) | Shipped and CI-gated; npm package not yet published | `zod` + `undici` + `ajv` + `canonicalize` (`hono` optional) | | Python | [`gen/python/`](https://github.com/RAMP-Protocol/protocol/tree/main/gen/python/) (types) + [`sdk/python/`](https://github.com/RAMP-Protocol/protocol/tree/main/sdk/python/) (hand-written SDK: `ramp_sdk`, `ramp_sdk.core`, `ramp_sdk.resolvers`, `ramp_sdk.client`, `ramp_sdk.sync`) | Shipped and CI-gated; PyPI package not yet published | `pydantic` + `httpx` + `cryptography` | +Every language also ships the account-setup role: `Register` and `GetAccountStatus` on the agent client (`Register`/`GetAccountStatus`, `register`/`getAccountStatus`, `register`/`get_account_status`), addressed by the Exchange domain on the request rather than by a configured origin, alongside the reader that fetches an Exchange's published registration requirements afresh on every read (`resolvers.NewWellKnownRequirementsReader`, `createWellKnownRequirementsReader`, `WellKnownRequirementsReader`). A payload the client's own schema pre-check refuses comes back carrying the same typed `RegistrationFailure` detail the Exchange would have attached, so one renderer serves both sides of the refusal. + Every language also ships the publisher role: a catalog client (`connect.NewCatalogClient`, `createCatalogClient`, `ramp_sdk.client.CatalogClient`) and the Exchange's two-tier entry pre-check (`helpers.ValidateResourceEntry`, `validateResourceEntry`, `validate_resource_entry`) — see [Source 7: CatalogService API Push](/components/content-ingestion/sources/#source-7-catalogservice-api-push). ### What Is Language-Specific vs Shared diff --git a/website/src/content/docs/getting-started/for-ai-agents.mdx b/website/src/content/docs/getting-started/for-ai-agents.mdx index 8e950db6..ad94b734 100644 --- a/website/src/content/docs/getting-started/for-ai-agents.mdx +++ b/website/src/content/docs/getting-started/for-ai-agents.mdx @@ -369,8 +369,9 @@ The core protocol is domain-agnostic. Extension profiles add domain-specific met ## Next Steps -1. Register your agent with a participating Exchange (`ExchangeService.Register`) — the Exchange creates your account and bills it based on your verified request signature -2. Install the SDK, declare your `supported_profiles`, and configure your budget -3. Replace your per-provider integrations with `client.Fetch()` +1. Register your agent with a participating Exchange — `Register` in Go, `register` in TypeScript and Python. The Exchange creates your account and bills it based on your verified request signature, never on anything you send. Address the call by naming the Exchange's bare domain on the request: the SDK resolves the endpoint from that Exchange's own manifest, reads the terms revision it publishes from a freshly fetched copy, and echoes it so the signature records what you accepted. +2. Check the account with `GetAccountStatus` / `getAccountStatus` before you transact. An empty account handle means you hold no account there yet; an account that exists but is not active is waiting for the operator to switch it on, and the remedy is to wait or contact them rather than to register again. An account is per-Exchange, so a client can hold several — the same client registers at each. +3. Install the SDK, declare your `supported_profiles`, and configure your budget +4. Replace your per-provider integrations with `client.Fetch()` For the full API, see [Agent SDK Overview](/components/agent-sdk/overview/) and [Fetch Flow](/components/agent-sdk/fetch-flow/). diff --git a/website/src/content/docs/protocol/exchange-manifest.mdx b/website/src/content/docs/protocol/exchange-manifest.mdx index 4a6d3f3c..1370ce81 100644 --- a/website/src/content/docs/protocol/exchange-manifest.mdx +++ b/website/src/content/docs/protocol/exchange-manifest.mdx @@ -180,6 +180,27 @@ is a denial-of-service vector and one that resolves references is an SSRF vector aimed wherever its author points. A publisher **MUST** satisfy every rule below, and a consumer **MUST** refuse a schema that does not. +The SDK is that consumer for a registering agent: it reads this block from a +**freshly fetched** manifest — never a cached one, because the terms digest beside +it cannot be checked for staleness locally — over its SSRF-guarded transport, since +the Exchange domain arrives on the request rather than from configuration. It then +compiles the schema under every rule here, and pre-checks the payload before +anything is signed. A schema it refuses is +reported as a verdict and the registration is sent anyway: refusing locally would +turn a rule about reading a third party's document into a denial of service against +the agent's own user, so the Exchange's own enforcement stays the deciding check. +The 16KB cap is measured over the bytes **as served**, in every language, so a +schema padded past it on the wire is refused at both ends rather than at one. + +A payload the pre-check does refuse comes back naming the offending members, as the +same typed `RegistrationFailure` detail an Exchange attaches when it refuses the same +payload — so an agent reads one shape whichever side declined, and its tooling needs +one renderer rather than two. The detail's `domain` names the client's own tier rather +than the Exchange, which never saw the request. Each entry addresses a member by RFC +6901 pointer, with the empty pointer addressing `registration_data` itself; the +constraint text beside it is developer-facing and validator-defined, so branch on the +pointer and the typed reason, never on that prose. + - **Self-contained.** Every `$ref`, `$dynamicRef` and `$recursiveRef` begins with `#`. A consumer never resolves a reference that leaves the document — doing so turns every reader into a fetch aimed at a URL the schema's author chose. diff --git a/website/src/content/docs/reference/changelog.mdx b/website/src/content/docs/reference/changelog.mdx index c34be853..37d6800e 100644 --- a/website/src/content/docs/reference/changelog.mdx +++ b/website/src/content/docs/reference/changelog.mdx @@ -8,6 +8,99 @@ and protocol history, see [`proto/CHANGELOG.md`](https://github.com/RAMP-Protoco ## Unreleased +**The SDK serves the account-setup role, in all three languages (no wire change; +conformance-affecting).** `Register` and `GetAccountStatus` land on the agent client — +`connect.Client` in Go, `createClient`'s face in TypeScript, `Client` and its blocking +twin in Python — under the same names, over the same signing transport, redirect +refusal, request-id and validate interceptors and read cap as the other agent verbs. +They sit on that client rather than on a fourth constructor because they are the same +party holding the same key, and they route as a usage report does: the destination is +read off the request's own `exchange`, resolved from that Exchange's own manifest, and +dialled on the guarded leg. An account is per-Exchange and which Exchange is the agent's +choice per call — a denial names where to register — so a configured origin would confine +registration to one Exchange. Neither message carries an idempotency key, so neither verb +takes call options; `ver` is stamped when empty; a request that names no bare-domain +recipient is refused before it is signed. The client-request corpus gains three rows, +replayed in all three languages. + +A new resolver face reads what one Exchange asks of a registration — the terms revision +submitting one accepts, and the schema its `registration_data` must match — from that +Exchange's own manifest, and it holds NO document cache. The contract requires a +registering client to read the terms digest from a freshly fetched manifest, and the +endpoint resolver is built out of exactly the mechanism that value may not touch, so a +face with no cache slot to reuse is what makes the rule structural. `Register` fills +`terms_digest` from it only when the caller left the field unset, applies the four +`registration_data` bounds before anything is signed, and pre-checks the payload against +the published schema — skipping the check, never the send, when that schema is one the +SDK refuses. The schema is measured over the bytes AS SERVED in every language, which the +two JSON ports reach by slicing the member out of the served body rather than +re-serialising a parsed value. + +The reader dials on the SSRF-guarded transport in all three languages, and is built once +with the client rather than once per registration: the Exchange domain comes off the +request, so it is an address another party chose, which is the provenance that takes the +guard. A deployment reaching a private Exchange injects its own transport or sets the +usual two environment flags. + +Two cross-cutting changes anyone re-pinning will see. The client's typed failure now +carries the peer's own developer message as a value — filled where the peer's own answer +is decoded, and empty otherwise: never a transport's synthesized status line, and never a +typed detail the SDK built itself, as the content leg does from an edge's refusal token. +So a consumer reads it instead of parsing it back out of a rendered error; it is +unbounded, and bounding it belongs to whoever displays it. And Python's `ClientConfig` gains `sign_window`, so the +RFC 9421 freshness knob sits at the tier Go and TypeScript already surface it at. + +A refusal the pre-check computes carries what it computed. The offending members travel as +a typed `RegistrationFailure` detail beside the sentence, so a consumer that renders the +Exchange's refusal and the client's own through one renderer reads one shape either way +rather than parsing members back out of prose. Its domain names the client's own tier: the +Exchange never saw the request. The field errors are compared across languages by POINTER +only — the constraint text beside each comes from a different JSON Schema library in each +port and the contract calls it validator-defined — and an empty pointer, which is how a +missing required member is reported, now renders without a leading separator instead of as +a member with no name. + +Two corrections to the content leg travel with it. The delivery edge's refusal tokens are +recorded in the proto beside each `RetrievalAuthFailureReason`, and the mapping is not +derivable — `expired` is `URL_EXPIRED`, `pop_expired` is `PROOF_EXPIRED`. Both JSON ports +had computed each name by uppercasing the token instead, which matches the record for two +of the eleven tokens, so nine refusals a real edge emits reached a TypeScript or Python +caller with no typed reason at all while any spelling that happened to match an enum +suffix was promoted. Both now carry the record. And an invalid-host refusal from an +injected requirements reader is classified `not_sent` in all three rather than retried in +two, which is what each port's own routing leg already answered. + +**The requirements seam gains a verdict for a document it cannot use (additive, no wire +change).** `resolvers.ErrManifestUnusable` / `ManifestUnusableError` / `ManifestUnusable` +says the document arrived and this reader cannot use it — final, not a transport failure, +because the next fetch returns the same bytes. It exists because the reader seam is +injectable: the SDK's own reader refuses three things and treats every other +disappointment as absence or as an outage, so an implementation stricter than it — one +validating the whole manifest, or refusing a version — held a permanent refusal the seam +reported as transient and a caller retried indefinitely. One consequence for anyone +re-pinning: this is a fourth verdict on that seam, and a classifier branching only on the +older three drops it into its transport-failure bucket and retries something that will +never succeed; add the new sentinel alongside. The SDK's own reader never returns it — +an off-spec optional member still reads as absent, and an undecodable document is still a +transport failure — so nothing changes for a caller that injects no reader. Python also +stops treating an unrelated `ValueError` as a verdict on both the routing and the account +leg: the invalid-host refusal is recognised by its wording now, as TypeScript already did, +so an injected seam raising `json.JSONDecodeError` is retryable there as it is in the +other two. + +A new shared corpus pins the details the SDK builds itself rather than receives — the two +above — over every recorded edge token, including the two that must stay untyped, replayed +in all three languages. Two conformance guards sit under it: one holds every committed +`ErrorDetail.domain` to the shape it claims (a `Service` suffix names a service the +contract defines, a bare noun names a tier that is not one), and one holds the corpus to +the token annotations in the `.proto` source. The first found a vector naming a +registration service the contract has never defined; that fixture is corrected. + +*Parity record:* six new mapped symbols and one Go-idiomatic exclusion; the reader's Go +factory folds into the Python class constructor as every other `NewX` does, so the +shrink-only allowlist baseline moves 16 → 17 as a reviewed bump under that one recorded +class. + **SDK endpoint resolvers enforce `WellKnownManifest.ver`, and the manifest version gets its own constant (SDK behaviour change; no wire change).** The `/.well-known/ramp.json` document's `ver` has stated a consumer-side rule since the diff --git a/website/src/content/docs/security/threat-model.mdx b/website/src/content/docs/security/threat-model.mdx index c0121b0b..cdf961a8 100644 --- a/website/src/content/docs/security/threat-model.mdx +++ b/website/src/content/docs/security/threat-model.mdx @@ -274,6 +274,19 @@ Line 3: Legal/contractual enforcement - **In-code (SDK, primary).** The SDKs ship a **safe-by-default** SSRF-guarded transport, and it is **two checks, not one**. The *address pin* is connection-level: it resolves-then-checks every candidate address against a shared reserved-range corpus and pins the checked IP to defeat rebinding, on a maintained HTTP client (`net/http` / `httpx` / `undici`). The *scheme allowlist* is http(s)-only on the initial URL and every redirect hop, and it is deliberately **not** part of the connector — by the time a connection-level hook runs, the URL has been reduced to a host and a port and the scheme is no longer visible. Go and Python wrap the transport, so one object carries both (`SSRFGuard` inside `NewGuardedTransport`, `ssrf_guard()` inside `guarded_client`). TypeScript composes `ssrfGuard()` as the undici connector and states the scheme rule beside it, in `requireScheme()`. Because both ship in the artifact, their correctness cannot depend on deployment — and a caller who replaces the dial replaces the address pin with it, so the scheme check sits **above** the dial in the client tier, where no injected transport can remove it. - **Egress boundary (operator, recommended defense-in-depth).** Operators SHOULD additionally run any role that resolves directories from request-supplied hosts behind **restricted egress** (an egress proxy or network policy that cannot route to RFC1918 / link-local / metadata) and **IMDSv2 with hop-limit 1**. This neutralizes the entire class regardless of any in-code gap — but it is *defense-in-depth, not a substitute* for the SDK guard, which remains the primary control for an OSS artifact that runs in deployments the maintainers do not control. +**Which fetches this covers, and where it does not reach yet.** The rule is the URL's **provenance**, not what is being fetched. A **request-derived** host is one another party chose — it arrives in a header, or on a signed message — so it takes the guarded transport; with one exception, named below, that is a gap rather than a choice. A **fixed, operator-chosen** address takes the plain transport deliberately: the operator rather than an attacker picked it, and an on-prem directory may legitimately sit on a private address a guard would refuse. Where each fetch stands today: + +| Fetch | URL comes from | Go | TypeScript | Python | +|---|---|---|---|---| +| WBA directory (`Signature-Agent`) | the request | guarded | guarded | guarded | +| Registration requirements (`RegisterRequest.exchange`) | the request | guarded | guarded | guarded | +| Exchange endpoint (`Offer.exchange`) | the request | guarded | **plain** | **plain** | +| Well-known JWKS | operator configuration | plain | plain | plain | + +The last row is the deliberate one — nobody but the operator can point that URL. The **third row is a known gap, not a decision**: that host comes off a signed offer, so by the rule above it belongs on the guarded transport in every language, and the two ports have not caught up. It is tracked separately. + +**What to do about it today.** TypeScript and Python require the application to supply the endpoint resolver — neither client builds one for you — so the remedy sits at a construction site you already own: pass the guarded transport explicitly, `guardedFetchFromEnv()` in TypeScript or `guarded_client()` in Python, rather than letting that resolver take its default. Conversely, a deployment that needs the guarded posture on a configured URL, or the plain one on a request-derived host, injects its own transport or sets `SKIP_SSRF` / `ALLOW_INSECURE`. + ### T-LIC-2: Silent restriction downgrade **Attack**: A publisher forbids training (`Restriction{kind: FUNCTION, prohibited: ["ai-train"]}`) but the consumer does not recognize a token, or the publisher forgot to mark the restriction binding. The consumer treats it as advisory and proceeds — silent infringement with plausible deniability.