Docs/publisher onboarding flow

docs/design-history.md

@@ -2958,6 +2958,12 @@ released call still compiles, as the client verbs' call options were added.
 
 ## A publisher proves its domain with the manifest it serves
 
+> **Superseded in part — the edge package no longer answers the manifest's address.** The
+> manifest is a file on the publisher's origin and the package passes the address through,
+> so the authorization below always lives in that file, never in the package's settings.
+> The proof, the catalog key and the RPCs stand as written. See "A publisher's documents
+> are files on its origin".
+
 The contract carries `RequestDomainVerification` and `ConfirmDomainVerification`, an
 ACME-style challenge in which the publisher places a token and the Exchange fetches it.
 The publisher guide does not walk through them, because the console publishers onboard
@@ -3266,3 +3272,32 @@ visible rather than silent. The chain cases the Go tests replay are kept outside
 `*-vectors.json` convention until those ports exist, because that convention is a promise
 of tri-language replay and the completeness gate enforces it without exemption.
 
+
+## A publisher's documents are files on its origin
+
+An edge package used to carry a publisher's manifest and RSL file in its settings and
+answer both addresses itself. That made two copies of the document that decides who may
+sell a publisher's pages: the file the console handed over and the values in the package.
+A publisher who edited the file changed nothing an agent or the Exchange read, and on
+CloudFront which copy answered depended on which cache behavior the function was attached
+to. A publisher's own `/rsl.txt` was hidden the same way, behind a 404 from the package.
+
+So the edge serves no publisher document. GET and HEAD of exactly `/.well-known/fora.json`
+and `/rsl.txt` go to the origin for every client, AI bots included, ahead of the bot check,
+the licensed-path check and signed URL verification, and the origin's answer comes back
+unchanged. A bot refused at an article is told to read that document, so it has to be able
+to, whatever it calls itself. The match is exact, so the rule frees nothing else.
+
+The reasons are the ones a single file gives. There is one copy, under the publisher's
+control. It behaves the same on every CDN and cache-behavior layout. A publisher adds an
+Exchange or changes a contributor by editing the file, with nothing to redeploy, and nothing
+drifts when another network is added. The cost is that a publisher has to place the file,
+which the console's last step asks for, and that an unreachable origin is a `502`: a
+document the edge made up would say nothing about the domain.
+
+Two things left the docs with it. `/provider/{domain}/fora.json` and
+`/provider/{domain}/rsl.txt` were Exchange endpoints for an edge to fetch, and an edge was
+their only reader. And no `fora-cli` ever drove the domain-verification challenge, and no
+edge store ever answered it. The RPC pair itself stays in the contract, for deployments
+that verify that way, and a publisher answering such a challenge places the token on its
+origin like any other file.

proto/CHANGELOG.md

@@ -1,5 +1,22 @@
 # FORA Protocol Changelog
 
+## Unreleased
+
+**The publisher docs describe the console's onboarding and the file on the publisher's
+origin (docs and comments only; no wire change).** The public onboarding page and the
+publisher guide walk the console's four steps: Your business, which is required, then
+Your domain, Your terms and Connect your site, which can be finished later and which
+every further domain takes again. Prices read "per fetch", and those two pages call the
+artifact the discovery file. The edge pages say a provider's `fora.json` and `rsl.txt`
+are files on its own origin: the edge forwards exactly those two paths for every client,
+ahead of all its checks, and keeps no copy, so the serving modes, the caching strategy
+and the settings that fed an edge-built document are gone. A domain is proven by the
+`fora.json` it serves. `RequestDomainVerification` and `ConfirmDomainVerification` stay in
+the contract, and their comments no longer name a `fora-cli` tool or an edge KV store. The
+Exchange pages lose the `/provider/{domain}/fora.json` and `/provider/{domain}/rsl.txt`
+endpoints, whose only reader was such an edge. The documentation conformance gate refuses
+the retired phrasings, and a website test holds both pages to the console's step names.
+
 ## v1.2.0
 
 ### Breaking against v1.1.0

proto/fora/v1/fora.proto

@@ -536,7 +536,7 @@ service ExchangeService {
   rpc DisputeTransaction(DisputeRequest) returns (DisputeResponse);
 
   // Request a domain verification challenge for provider onboarding.
-  // Used by fora-cli to prove domain control before pushing signing keys.
+  // Proves domain control before signing keys are pushed.
   // Follows the ACME HTTP-01 pattern (Let's Encrypt).
   rpc RequestDomainVerification(DomainVerificationRequest) returns (DomainVerificationChallenge);
 
@@ -4751,13 +4751,12 @@ message DisputeResponse {
 //
 // ACME HTTP-01 style domain verification for provider onboarding.
 // Proves that the caller controls the provider domain before accepting
-// signing keys or configuration changes. Used by fora-cli.
+// signing keys or configuration changes.
 //
 // Flow:
 //   1. Provider calls RequestDomainVerification with their domain
 //   2. Exchange returns a challenge token
 //   3. Provider places the token at {domain}/.well-known/fora-verify/{token}
-//      (edge function can auto-serve this from KV store)
 //   4. Provider calls ConfirmDomainVerification
 //   5. Exchange fetches the challenge URL to verify domain control
 //   6. If verified: Exchange accepts subsequent operations for this domain

scripts/check-doc-conformance.sh

@@ -134,6 +134,16 @@ patterns=(
   'estimateTolerance' 'EstimateToleranceBps' 'MeteredSettlement'
   'metered_settlement' 'meteredSettlement' 'SettleMeteredUsage' 'settle_metered_usage'
   'settleMeteredUsage' 'settling-a-metered-purchase'
+  # The edge serves no publisher document: a provider's fora.json and rsl.txt are
+  # files on its origin, and the edge forwards both addresses for every client. So
+  # no serving mode picks a source for them, no Exchange endpoint hands them to an
+  # edge, and no edge store answers the domain-verification challenge. These are the
+  # retired phrasings of those claims, with the CLI that was said to drive them.
+  # fora-verify is NOT listed: the contract's domain-verification messages name that
+  # path, and the RPC pair stays. The /provider/ pattern names the two documents
+  # because /provider/{domain}/transactions is a live audit route.
+  'fora-cli' 'auto-serve' 'serving_mode' 'servingMode' 'exchange_api_url' 'exchangeApiUrl'
+  '/provider/\{domain\}/(fora\.json|rsl\.txt)'
 )
 
 # Files where naming a removed identifier is legitimate (they record history).

website/src/content/docs/architecture/deployment-models.mdx

@@ -7,17 +7,9 @@ Three deployment models cover the full provider spectrum: enterprise providers w
 
 ## Provider Onboarding: Domain Verification (v1.0)
 
-Before any deployment model, the provider must prove it controls its domain. With the contract's domain-verification RPCs, that is an ACME HTTP-01 style challenge, handled by `fora-cli`:
+Before any deployment model, the provider must prove it controls its domain. The proof is the `fora.json` the domain serves from the provider's own origin. The Exchange reads that file at registration, on every check of a domain being verified and at every catalog push, so the proof keeps being renewed: a file that stops listing the Exchange stops the domain's pushes. [Publisher Onboarding](/getting-started/publisher-onboarding/) describes that path through a publisher console.
 
-1. Provider runs `fora-cli verify --domain techcrunch.com --exchange exchange.ssp-alpha.com`
-2. CLI calls `RequestDomainVerification` -- Exchange returns a challenge token
-3. CLI places the token at `{domain}/.well-known/fora-verify/{token}` (the edge function can auto-serve this from KV store)
-4. CLI calls `ConfirmDomainVerification` -- Exchange fetches the challenge URL to verify domain control
-5. If verified: Exchange accepts subsequent operations (signing key registration, catalog pushes) for this domain
-
-The Exchange also checks `fora.json` to verify it is listed as an authorized exchange for this provider -- providing double protection.
-
-An Exchange can instead take the `fora.json` the domain serves as that proof, as the Exchange behind a publisher console does; [Publisher Onboarding](/getting-started/publisher-onboarding/) describes that path.
+The contract also carries `RequestDomainVerification` and `ConfirmDomainVerification`, for deployments that verify a domain with a challenge instead (see the [contract reference](/reference/proto-fora/#messages--domain-verification)).
 
 ### catalog_contributors in fora.json
 
@@ -43,22 +35,24 @@ The Exchange operator deploys and maintains the FORA edge function on the provid
 ### How It Works
 
 1. Provider signs a contract with the Exchange operator and grants CDN access (e.g., AWS IAM role for CloudFront, Cloudflare API token for Workers)
-2. Exchange operator runs domain verification on behalf of the provider (`fora-cli verify`)
-3. Exchange operator deploys the edge function to the provider's CDN
-4. Exchange operator configures: `fora.json` (including `catalog_contributors`), `rsl.txt` generation from the Exchange, bot detection rules
-5. Exchange operator maintains, updates, and monitors the edge function on an ongoing basis
+2. Exchange operator generates the provider's `fora.json` (including `catalog_contributors`) and, where the provider has none, its `rsl.txt`
+3. Provider publishes those files on its own origin, which proves it controls the domain
+4. Exchange operator deploys the edge function to the provider's CDN
+5. Exchange operator configures bot detection rules
+6. Exchange operator maintains, updates, and monitors the edge function on an ongoing basis
 
 ### Provider Actions
 
 - Sign contract with Exchange operator
 - Grant CDN role/access to the Exchange operator
-- Nothing else -- the Exchange operator handles all FORA infrastructure
+- Publish the generated files on its own origin
+- The Exchange operator handles the rest of the FORA infrastructure
 
 ### Exchange Operator Responsibilities
 
 - Deploy edge function to provider's CDN account
-- Generate and serve `fora.json` from Exchange tenant config
-- Generate and serve `rsl.txt` from Exchange catalog
+- Generate `fora.json` from Exchange tenant config, for the provider to publish on its origin
+- Generate `rsl.txt` from Exchange catalog, where the provider has none of its own
 - Configure bot detection rules (User-Agent matching, behavioral signals)
 - Rotate signing keys on schedule (default: 90 days)
 - Monitor edge function health, latency, and error rates
@@ -81,7 +75,7 @@ The provider deploys the edge function themselves using tooling provided by FORA
    ```bash
    npx @fora/edge-deploy --cdn cloudfront --exchange exchange.ssp.com
    ```
-3. Edge function pulls `fora.json` and `rsl.txt` from the Exchange, caches in KV/edge storage
+3. Provider publishes `fora.json`, and `rsl.txt` if it has one, on its own origin; the edge function passes both through to it
 4. Provider manages their own CDN access and can customize bot detection rules
 
 ### Supported CDNs
@@ -149,15 +143,15 @@ For CDNs with limited compute capabilities, the edge function can be reduced to
 
 1. **Signed URL verification**: verify the Ed25519 signature over the canonical URL -- valid? pass through : return 403. A CloudFront distribution verifies its RSA signed URLs natively, so there the function skips this step.
 2. **Bot detection**: Check `User-Agent` for known AI crawler strings -- match? return 403 with `X-Content-Rules` header
-3. **Serve `fora.json`**: Return `/.well-known/fora.json` inline or proxied from the Exchange
+3. **Let `fora.json` through**: Leave `/.well-known/fora.json` to the origin, ahead of the bot check, so a refused bot can read it
 
 ### Minimum Viable Decision Table
 
 ```
 Request arrives at edge:
 
-1. Path = /.well-known/fora.json?
-   YES -> Return inline fora.json (or proxy from Exchange)
+1. Path = /.well-known/fora.json or /rsl.txt?
+   YES -> Pass through to origin
 
 2. Has signed URL params (sig, exp)?
    YES -> Verify signature
@@ -170,4 +164,4 @@ Request arrives at edge:
 4. Default -> Pass through to origin (non-bot, non-protected)
 ```
 
-Step 1 needs a runtime that can run code: Cloudflare Workers and Fastly Compute verify the signature themselves, and a CloudFront distribution verifies its own RSA signed URLs without any function at all. A declarative rule language -- Fastly VCL, Akamai Property Manager -- cannot verify a signature, so it can serve `fora.json` and refuse known crawlers but cannot admit a paid request.
+Step 1 needs a runtime that can run code: Cloudflare Workers and Fastly Compute verify the signature themselves, and a CloudFront distribution verifies its own RSA signed URLs without any function at all. A declarative rule language -- Fastly VCL, Akamai Property Manager -- cannot verify a signature, so it can refuse known crawlers but cannot admit a paid request.

website/src/content/docs/architecture/production-architecture.mdx

@@ -91,7 +91,7 @@ Dispute lifecycle: `FILED` -> `AUTO_RESOLVED` / `EVIDENCE_NEEDED` -> `UNDER_REVI
 
 ### Provider Domain Verification (v1.0)
 
-The `RequestDomainVerification` and `ConfirmDomainVerification` RPCs provide ACME HTTP-01 style domain verification for provider onboarding. Providers prove domain control before the Exchange accepts signing keys or configuration changes. The WBA directory at `/.well-known/http-message-signatures-directory` enables verification vendors to publish their Ed25519 keys (in its `keys`) for attestation validation. FORA has no dedicated verifier role; a verifier publishes under its operator's role (`ROLE_PUBLISHER` for a self-attesting publisher, `ROLE_EXCHANGE` for a standalone vendor).
+A provider proves it controls its domain with the `fora.json` it serves, which the Exchange reads before it accepts catalog pushes or configuration changes for that domain, and again at each push. The contract also carries the `RequestDomainVerification` and `ConfirmDomainVerification` RPCs, for deployments that verify a domain with a challenge instead. The WBA directory at `/.well-known/http-message-signatures-directory` enables verification vendors to publish their Ed25519 keys (in its `keys`) for attestation validation. FORA has no dedicated verifier role; a verifier publishes under its operator's role (`ROLE_PUBLISHER` for a self-attesting publisher, `ROLE_EXCHANGE` for a standalone vendor).
 
 ### Discovery: Three Complementary Standards
 

website/src/content/docs/components/content-ingestion/rsl-generation.mdx

@@ -5,12 +5,12 @@ description: "How the Exchange generates standards-compliant rsl.txt from the co
 
 ## Exchange as RSL Producer
 
-The Exchange is both an RSL consumer (ingesting provider rsl.txt to build the catalog) and an RSL producer (generating rsl.txt for providers who don't have one). This enables a zero-configuration provider onboarding path: the Exchange operator sets tenant defaults, and the Exchange generates a standards-compliant rsl.txt that the edge function can serve.
+The Exchange is both an RSL consumer (ingesting provider rsl.txt to build the catalog) and an RSL producer (generating rsl.txt for providers who don't have one). This enables a zero-configuration provider onboarding path: the Exchange operator sets tenant defaults, and the Exchange generates a standards-compliant rsl.txt that the provider publishes on its own origin.
 
 ```mermaid
 sequenceDiagram
-    participant PUB as Provider (no rsl.txt)
     participant MP as Exchange
+    participant PUB as Provider's Origin (no rsl.txt yet)
     participant EDGE as Edge Function
     participant AGENT as AI Agent
 
@@ -19,11 +19,13 @@ sequenceDiagram
     MP->>MP: Build catalog from tenant defaults + sitemap
     MP->>MP: Generate rsl.txt from catalog
 
-    AGENT->>PUB: GET /rsl.txt
-    PUB->>EDGE: Viewer request
-    EDGE->>MP: GET /provider/{domain}/rsl.txt
-    MP-->>EDGE: Generated RSL XML
-    EDGE-->>AGENT: rsl.txt (cached at edge)
+    MP-->>PUB: Generated RSL XML, handed over by the operator
+    Note over PUB: Provider publishes it at /rsl.txt
+
+    AGENT->>EDGE: GET /rsl.txt
+    EDGE->>PUB: Forward the request
+    PUB-->>EDGE: rsl.txt
+    EDGE-->>AGENT: The origin's answer, unchanged
 ```
 
 ## RSL Generator
@@ -137,13 +139,7 @@ For a tenant with defaults "flat, $0.05, permits ai-input + search, prohibits ai
 
 ## Serving the Generated RSL
 
-The Exchange exposes the generated RSL via an HTTP endpoint:
-
-```
-GET /provider/{domain}/rsl.txt
-```
-
-The provider's edge function fetches and caches this, serving it at the standard `/rsl.txt` path. Cache TTL matches the ingestion refresh interval (default: 24h for full refresh, 5m for incremental).
+The provider publishes the generated document at the standard `/rsl.txt` path on its own origin, and replaces it when the catalog changes. The edge function [passes that address through](/components/edge-function/overview/#forajson-and-rsltxt-pass-through) for every client, so an agent reads the file the provider published.
 
 ## FORA Sitemap Namespace (XSD)
 

website/src/content/docs/components/edge-function/bot-detection.mdx

@@ -147,7 +147,7 @@ Target: **100 requests/minute per IP** before throttling.
 
 **Level 2: Per-IP rate limiting on fora.json**
 
-fora.json is cacheable, but an aggressive agent might poll it. Rate limit to **10 requests/minute per IP** and rely on `Cache-Control` for normal operation.
+fora.json is cacheable, but an aggressive agent might poll it. Rate limit to **10 requests/minute per IP** and rely on `Cache-Control` for normal operation. This is a platform rule in front of the edge function, which only forwards the path to the origin. An Exchange reads the file at registration, on every check of a domain being verified and at every catalog push, so keep the limit above that rate.
 
 Rate limiting implementation:
 - **Cloudflare**: Use `request.cf.botManagement` and Cloudflare Rate Limiting rules (not in Worker code -- use platform rate limiting for efficiency).