Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
26 changes: 26 additions & 0 deletions changelog.d/20260930_lab-5906.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,26 @@
### Interop mode — `..` is forbidden inside a segment (LAB-5906)

- [`spec/interop-mode.md` → Segment grammar](spec/interop-mode.md#segment-grammar):
a segment (`namespace` or `operation`) MUST NOT contain `..`. The segment pattern
admits it (`a..b`, `users.v1..beta`), but the server rejects `..` anywhere in a key
([cache-key-format.md → Server-Side Requirements](spec/cache-key-format.md#server-side-requirements),
the Traversal row), so every SDK minted a key that CachekitIO answered with `400` on
every request, while the same key worked on Redis and file backends. The pattern
itself is unchanged and stays a plain regex; the rule is a separate substring check.
A lone `.` stays valid. **Breaking for any deployment whose interop namespace or
operation contains `..`, on any backend:** it now raises at decoration /
registration time; migrate by renaming the segment (a full cache miss for the keys
it names).
- [`test-vectors/interop-mode.json`](test-vectors/interop-mode.json) 1.2.0: two error
vectors (`reject_double_dot_namespace`, namespace `a..b`; `reject_double_dot_operation`,
operation `x..`, with the `..` at the end of the segment) and one key vector
(`lone_dots_stay_valid`: namespace `app.`, operation `users.fetch.by_id`), the first key
vector with a `.` in a segment. Counts: 35 key, 13 error.
- The status banner and SaaS Considerations no longer name `..` as an exception to the
grammar being a subset of what the server accepts.
- SDK feature matrix: the "Test vectors in CI" cells link the SDK PRs that vendor fixture
1.2.0, none released yet. The Python cell now says fixture 1.1.0 ships in PyPI 0.20.0.
- `tools/interop-reference.py` rejects a `..` segment, and its self-check asserts that
both new error vectors match `segment_pattern`, so they exercise the new rule rather
than the pattern. `tools/interop-crosscheck.mjs` hard-codes the rule rather than
reading it from the fixture, as it does for the reserved namespaces.
2 changes: 1 addition & 1 deletion sdk-feature-matrix.md
Original file line number Diff line number Diff line change
Expand Up @@ -299,7 +299,7 @@ its spec:
| AAD v0x03 | ✅ Compliant (5 components — every auto serializer appends `original_type`; interop mode is the sole 4-component path) | ✅ Compliant (4 components) | ✅ Compliant (4 components) | ❌ Not implemented |
| SaaS API | ✅ Compliant (CachekitIO backend) | ✅ Compliant (CachekitIO backend) | ✅ Compliant | ❌ Not implemented |
| SaaS API — cache-key path encoding ([spec](spec/saas-api.md#cache-key-path-encoding)) | ✅ Compliant on `main`, unreleased ([cachekit-py#364](https://github.com/cachekit-io/cachekit-py/pull/364)); rules 1/3/4 since 0.18.0 ([cachekit-py#279](https://github.com/cachekit-io/cachekit-py/pull/279)) | ✅ Compliant on `main`, unreleased ([cachekit-rs#76](https://github.com/cachekit-io/cachekit-rs/pull/76)) | ✅ Compliant on `main`, unreleased ([cachekit-ts#118](https://github.com/cachekit-io/cachekit-ts/pull/118)) | ❌ Not implemented |
| Test vectors in CI¹⁶ | ✅ interop/v1 (full set, incl. AAD + encryption through the real stack) — fixture 1.1.0 (`ns`/`nsapi` namespace reservation) in [cachekit-py#350](https://github.com/cachekit-io/cachekit-py/pull/350), unreleased; `decode-bounds.json` vendored + CI-executed since [cachekit-py#276](https://github.com/cachekit-io/cachekit-py/pull/276) (LAB-2503); `1.1.0` in [cachekit-py#363](https://github.com/cachekit-io/cachekit-py/pull/363), unreleased, asserting the structural guard's own error for every reject vector at every payload read path (the [Decode bounds](spec/interop-mode.md#decode-bounds) MUST) — ⚠️ except the envelope entry point (`ByteStorage::retrieve`), which has no guard to assert until cachekit-py moves to a core release carrying [cachekit-core#80](https://github.com/cachekit-io/cachekit-core/pull/80) (see the Wire format row); `path-encoding.json` vendored + CI-executed since [cachekit-py#364](https://github.com/cachekit-io/cachekit-py/pull/364), unreleased | ✅ interop/v1 (full set) since [#33](https://github.com/cachekit-io/cachekit-rs/pull/33) — fixture 1.1.0 in [cachekit-rs#89](https://github.com/cachekit-io/cachekit-rs/pull/89), unreleased; `decode-bounds.json` vendored + CI-executed since [cachekit-rs#73](https://github.com/cachekit-io/cachekit-rs/pull/73) (LAB-2503; default CI green on `main`); `1.1.0` in [cachekit-rs#93](https://github.com/cachekit-io/cachekit-rs/pull/93), unreleased, asserting the structural guard's own `decode bound:` error for every reject vector through both decoders and `get` / `interop_get` / `interop_get_swr` — the guard enforces depth itself since that PR (no envelope entry point¹⁵) | ✅ interop/v1 (full set, incl. its key vectors) + inline Python-generated AAD-construction and encryption (decrypt-Python-ciphertext) vectors — fixture 1.1.0 in [cachekit-ts#143](https://github.com/cachekit-io/cachekit-ts/pull/143), unreleased; decode bounds enforced ([#112](https://github.com/cachekit-io/cachekit-ts/pull/112)); `decode-bounds.json` vendored + CI-executed since [cachekit-ts#121](https://github.com/cachekit-io/cachekit-ts/pull/121) (LAB-2737), `1.1.0` in [cachekit-ts#152](https://github.com/cachekit-io/cachekit-ts/pull/152), unreleased, asserting the guard error each vector trips (pre-scan, or the event size cap ahead of it) — ⚠️ except the envelope entry point (`unpack` in `cachekit-core-ts` and `cachekit-core-wasm`), which has no guard to assert until both bindings move to a core release carrying [cachekit-core#80](https://github.com/cachekit-io/cachekit-core/pull/80) and the vectors run through `unpack` (see the Wire format row); `path-encoding.json` vendored + CI-executed since [cachekit-ts#118](https://github.com/cachekit-io/cachekit-ts/pull/118), unreleased | ⚠️ Pending |
| Test vectors in CI¹⁶ | ✅ interop/v1 (full set, incl. AAD + encryption through the real stack) — fixture 1.1.0 (`ns`/`nsapi` namespace reservation) since PyPI 0.20.0 ([cachekit-py#350](https://github.com/cachekit-io/cachekit-py/pull/350)); fixture 1.2.0 (no `..` in a segment) in [cachekit-py#391](https://github.com/cachekit-io/cachekit-py/pull/391), unreleased; `decode-bounds.json` vendored + CI-executed since [cachekit-py#276](https://github.com/cachekit-io/cachekit-py/pull/276) (LAB-2503); `1.1.0` in [cachekit-py#363](https://github.com/cachekit-io/cachekit-py/pull/363), unreleased, asserting the structural guard's own error for every reject vector at every payload read path (the [Decode bounds](spec/interop-mode.md#decode-bounds) MUST) — ⚠️ except the envelope entry point (`ByteStorage::retrieve`), which has no guard to assert until cachekit-py moves to a core release carrying [cachekit-core#80](https://github.com/cachekit-io/cachekit-core/pull/80) (see the Wire format row); `path-encoding.json` vendored + CI-executed since [cachekit-py#364](https://github.com/cachekit-io/cachekit-py/pull/364), unreleased | ✅ interop/v1 (full set) since [#33](https://github.com/cachekit-io/cachekit-rs/pull/33) — fixture 1.1.0 in [cachekit-rs#89](https://github.com/cachekit-io/cachekit-rs/pull/89), unreleased; fixture 1.2.0 in [cachekit-rs#98](https://github.com/cachekit-io/cachekit-rs/pull/98), unreleased; `decode-bounds.json` vendored + CI-executed since [cachekit-rs#73](https://github.com/cachekit-io/cachekit-rs/pull/73) (LAB-2503; default CI green on `main`); `1.1.0` in [cachekit-rs#93](https://github.com/cachekit-io/cachekit-rs/pull/93), unreleased, asserting the structural guard's own `decode bound:` error for every reject vector through both decoders and `get` / `interop_get` / `interop_get_swr` — the guard enforces depth itself since that PR (no envelope entry point¹⁵) | ✅ interop/v1 (full set, incl. its key vectors) + inline Python-generated AAD-construction and encryption (decrypt-Python-ciphertext) vectors — fixture 1.1.0 in [cachekit-ts#143](https://github.com/cachekit-io/cachekit-ts/pull/143), unreleased; fixture 1.2.0 in [cachekit-ts#164](https://github.com/cachekit-io/cachekit-ts/pull/164), unreleased; decode bounds enforced ([#112](https://github.com/cachekit-io/cachekit-ts/pull/112)); `decode-bounds.json` vendored + CI-executed since [cachekit-ts#121](https://github.com/cachekit-io/cachekit-ts/pull/121) (LAB-2737), `1.1.0` in [cachekit-ts#152](https://github.com/cachekit-io/cachekit-ts/pull/152), unreleased, asserting the guard error each vector trips (pre-scan, or the event size cap ahead of it) — ⚠️ except the envelope entry point (`unpack` in `cachekit-core-ts` and `cachekit-core-wasm`), which has no guard to assert until both bindings move to a core release carrying [cachekit-core#80](https://github.com/cachekit-io/cachekit-core/pull/80) and the vectors run through `unpack` (see the Wire format row); `path-encoding.json` vendored + CI-executed since [cachekit-ts#118](https://github.com/cachekit-io/cachekit-ts/pull/118), unreleased | ⚠️ Pending |
| Interop mode ([spec](spec/interop-mode.md), opt-in) | ✅ Released — PyPI 0.14.0+¹⁷ ([#220](https://github.com/cachekit-io/cachekit-py/pull/220)) | ✅ Released — crates.io 0.4.0+ ([#33](https://github.com/cachekit-io/cachekit-rs/pull/33)) | ✅ Released — npm 0.1.3+ ([#71](https://github.com/cachekit-io/cachekit-ts/pull/71)) | ❌ Not implemented |

> [!NOTE]
Expand Down
26 changes: 17 additions & 9 deletions spec/interop-mode.md
Original file line number Diff line number Diff line change
Expand Up @@ -13,8 +13,7 @@
> each registry or the [SDK feature matrix](../sdk-feature-matrix.md#compliance-status) for current versions.
> Server-side: the CachekitIO validator accepts interop-format keys
> (`{namespace}:{operation}:{args_hash}` scopes to the `default` namespace;
> see [cache-key-format.md → Server-Side Requirements](cache-key-format.md#server-side-requirements)),
> except a key with `..` in a segment ([SaaS Considerations](#saas-considerations)).
> see [cache-key-format.md → Server-Side Requirements](cache-key-format.md#server-side-requirements)).
> Design discussion: [Issue #1](https://github.com/cachekit-io/protocol/issues/1) ·
> Test vectors: [`test-vectors/interop-mode.json`](../test-vectors/interop-mode.json) ·
> Reference implementation: [`tools/interop-reference.py`](../tools/interop-reference.py)
Expand Down Expand Up @@ -125,6 +124,17 @@ reservation is exact-match and namespace-only — `nsapix` is a valid namespace,
and `nsapi` are valid operations. The `reject_reserved_namespace_*` error vectors and the
`reservation_scope` key vector pin it.

A segment (`namespace` or `operation`) additionally MUST NOT contain `..`: the server
rejects `..` anywhere in a key
([cache-key-format.md → Server-Side Requirements](cache-key-format.md#server-side-requirements),
the Traversal row), so such a key would fail on every CachekitIO request. The pattern
admits `..`, so this is a separate check beside it; the pattern stays a plain regex
without lookahead. SDKs reject a `..` segment like a reserved namespace: at decoration /
registration time, on every backend. A lone `.` stays valid, including at the end of a
segment, and so do dots that are not adjacent (`app.`, `app.v1`, `users.fetch.by_id`). The
`reject_double_dot_*` error vectors (`..` inside and at the end of a segment) and the
`lone_dots_stay_valid` key vector pin it.

> [!WARNING]
> **Full-string means full-string.** In Python, `re.match` with a `$` anchor still
> accepts a trailing newline (`"users\n"` passes) — use `re.fullmatch`. A segment
Expand Down Expand Up @@ -399,10 +409,8 @@ isolation comes from authentication, not key parsing).
> accepts interop-format keys; see
> [cache-key-format.md → Server-Side Requirements](cache-key-format.md#server-side-requirements).
> The interop segment grammar (lowercase, no `:` beyond the two delimiters, no `/`,
> max 194 chars, no reserved namespace) is deliberately a subset of what the
> security-only validator accepts, with one known exception: the grammar admits `..`
> inside a segment, and the validator rejects `..` anywhere in a key (the Traversal
> row), so such a key fails with `400`.
> no `..`, max 194 chars, no reserved namespace) is deliberately a subset of what the
> security-only validator accepts.

---

Expand Down Expand Up @@ -439,7 +447,7 @@ const getUser = cache.wrap(fetchUser, {
An SDK implementation of interop mode MUST:

1. Require explicit `namespace` and `operation`, validated against the segment grammar
(including the reserved namespaces `ns` and `nsapi`).
(including the reserved namespaces `ns` and `nsapi`, and no `..` in either segment).
2. Build the canonical argument array per the binding rules (named→positional,
defaults applied where introspectable).
3. Normalize and encode per this spec; reject out-of-model values with an error.
Expand Down Expand Up @@ -561,11 +569,11 @@ not re-litigated by accident.

| Group | Count | Verifies |
| :--- | :---: | :--- |
| `key_vectors` | 34 | Canonical argument bytes (exact hex), args hash, full key — the `2.0`≡`2` collapse pair, supplementary-plane key sorting, heterogeneous and mixed-sign sets (byte order ≠ natural order), set dedupe (`{2, 2.0}` → `[2]`), datetime edge cases incl. pre-epoch, both collapse-range endpoints, every `*16`-tier width boundary (uint/int ladders, str/bin/array/map headers, root array16), and the reservation's exact-match, namespace-only scope (`nsapix` namespace, `nsapi` operation) |
| `key_vectors` | 35 | Canonical argument bytes (exact hex), args hash, full key — the `2.0`≡`2` collapse pair, supplementary-plane key sorting, heterogeneous and mixed-sign sets (byte order ≠ natural order), set dedupe (`{2, 2.0}` → `[2]`), datetime edge cases incl. pre-epoch, both collapse-range endpoints, every `*16`-tier width boundary (uint/int ladders, str/bin/array/map headers, root array16), the reservation's exact-match, namespace-only scope (`nsapix` namespace, `nsapi` operation), and lone dots, which stay valid (`app.` namespace, `users.fetch.by_id` operation) |
| `value_vectors` | 4 | Plain-MessagePack value bytes (exact hex), float64 preservation in the value profile, temporal sentinel maps |
| `aad_vectors` | 1 | AAD v0x03 bytes over an interop key (`format=msgpack`, `compressed=False`) |
| `encryption_vectors` | 1 | Full HKDF-SHA256 → AES-256-GCM round-trip over plain-msgpack plaintext with the interop AAD (fixed nonce; decrypt-verified) |
| `error_vectors` | 11 | Inputs that MUST be rejected (NaN, +Inf and −Inf as independent vectors, int overflow/underflow, naive datetime, bad segments incl. trailing newline, the reserved namespaces `ns` and `nsapi`). The `error` text is a maintainer note, not a normative message |
| `error_vectors` | 13 | Inputs that MUST be rejected (NaN, +Inf and −Inf as independent vectors, int overflow/underflow, naive datetime, bad segments incl. trailing newline, the reserved namespaces `ns` and `nsapi`, `..` in either segment). The `error` text is a maintainer note, not a normative message |

[`test-vectors/decode-bounds.json`](../test-vectors/decode-bounds.json) pins the
[Decode bounds](#decode-bounds); `tools/decode-bounds-reference.py verify` checks it,
Expand Down
30 changes: 28 additions & 2 deletions test-vectors/interop-mode.json
Original file line number Diff line number Diff line change
@@ -1,12 +1,12 @@
{
"version": "1.1.0",
"version": "1.2.0",
"spec": "spec/interop-mode.md",
"generator": "tools/interop-reference.py (CPython stdlib)",
"cross_checked_by": "tools/interop-crosscheck.mjs (independent encoder + @noble/hashes blake2b + WebCrypto HKDF/AES-GCM)",
"hash_algorithm": "blake2b-256 (digest_size=32, unkeyed) over canonical MessagePack of the flat argument array",
"key_format": "{namespace}:{operation}:{args_hash}",
"segment_pattern": "^[a-z0-9][a-z0-9._-]{0,63}$",
"segment_pattern_note": "Full-string match REQUIRED (Python: re.fullmatch, not re.match \u2014 $ matches before a trailing newline). namespace additionally MUST NOT be exactly 'ns' or 'nsapi' (reserved: the server parses those key prefixes). The reservation is namespace-only; operation has no reserved values.",
"segment_pattern_note": "Full-string match REQUIRED (Python: re.fullmatch, not re.match \u2014 $ matches before a trailing newline). namespace additionally MUST NOT be exactly 'ns' or 'nsapi' (reserved: the server parses those key prefixes). The reservation is namespace-only; operation has no reserved values. Neither segment may contain '..' (the pattern admits it; the server rejects '..' anywhere in a key). A lone '.' stays valid.",
"width_coverage_note": "All *16 header boundaries (uint/int widths, str8->str16, bin8->bin16, fixarray->array16, fixmap->map16, including the root argument array) are pinned by vectors. The *32 tier (str32/bin32/array32/map32, >=64 KiB or >=65536 elements) is normative and implemented by both tools but untested-by-design: fixture blobs that size would bloat the file without exercising different logic (same length-prefix code path, wider field).",
"error_vectors_note": "The 'error' field is a human-readable reason for maintainers. Conformance means the input MUST be rejected with an error; the message text is not normative.",
"tagged_json": {
Expand Down Expand Up @@ -605,6 +605,18 @@
"canonical_args_hex": "9101",
"args_hash": "405f09a3617bcc1425ea95b9840d9c2713e3ecebd5a3227abc599317b732e21a",
"expected_key": "nsapix:nsapi:405f09a3617bcc1425ea95b9840d9c2713e3ecebd5a3227abc599317b732e21a"
},
{
"name": "lone_dots_stay_valid",
"description": "Only '..' is forbidden: a lone trailing '.' (namespace 'app.') and non-adjacent dots (operation 'users.fetch.by_id') stay valid",
"namespace": "app.",
"operation": "users.fetch.by_id",
"args": [
1
],
"canonical_args_hex": "9101",
"args_hash": "405f09a3617bcc1425ea95b9840d9c2713e3ecebd5a3227abc599317b732e21a",
"expected_key": "app.:users.fetch.by_id:405f09a3617bcc1425ea95b9840d9c2713e3ecebd5a3227abc599317b732e21a"
}
],
"value_vectors": [
Expand Down Expand Up @@ -738,6 +750,20 @@
"operation": "users.fetch_by_id",
"args": [],
"error": "namespace 'nsapi' is reserved: the server parses a key starting 'nsapi:' as namespace-prefixed (rejected whatever the operation, including one the server would 400 on for its '.')"
},
{
"name": "reject_double_dot_namespace",
"namespace": "a..b",
"operation": "get_user",
"args": [],
"error": "namespace must not contain '..': the pattern admits it, but the server rejects '..' anywhere in a key"
},
{
"name": "reject_double_dot_operation",
"namespace": "users",
"operation": "x..",
"args": [],
"error": "operation must not contain '..': the pattern admits it, but the server rejects '..' anywhere in a key (here at the end of the segment, which a check that skips the last pair misses)"
}
],
"aad_vectors": [
Expand Down
8 changes: 5 additions & 3 deletions tools/interop-crosscheck.mjs
Original file line number Diff line number Diff line change
Expand Up @@ -270,12 +270,14 @@ const vectorsPath = process.argv[2] ?? join(here, "..", "test-vectors", "interop
const doc = JSON.parse(readFileSync(vectorsPath, "utf8"));

// Segment grammar: the fixture's pattern governs both segments; the reserved
// namespaces are hard-coded from the spec, not read from the fixture, so the
// reservation is checked by a second implementation rather than echoed back.
// namespaces and the `..` ban are hard-coded from the spec, not read from the
// fixture, so both rules are checked by a second implementation rather than
// echoed back.
const segmentRe = new RegExp(doc.segment_pattern, "u");
const RESERVED_NAMESPACES = new Set(["ns", "nsapi"]);
const segmentValid = (segment) => segmentRe.test(segment) && !segment.includes("..");
const segmentsValid = (namespace, operation) =>
segmentRe.test(namespace) && segmentRe.test(operation) && !RESERVED_NAMESPACES.has(namespace);
segmentValid(namespace) && segmentValid(operation) && !RESERVED_NAMESPACES.has(namespace);

let failures = 0;
const check = (name, kind, expected, actual) => {
Expand Down
Loading
Loading