Skip to content

regex parity: Python surfaces still disagree with validate_json after #388 #405

Closed

Description

@altrudev

Problem

#379 / #388 fixed the schema-verifier side of TRACE's ECMA-262 regex semantics by adapting the shipped patterns in validate.py before Python evaluates them.

Current main still has other producer/helper surfaces that compile the same textual rules directly with Python re:

  • models.py carries the raw subject pattern for Pydantic;
  • content_marking.py compiles raw _SUBJECT_RE and _DIGEST_RE;
  • adapters/sandbox.py compiles raw _SUBJECT_RE and _DIGEST_RE.

So the repository can now hold the same pattern string while applying different acceptance semantics depending on which API surface is used.

That matters because the parity test can report agreement on the string while runtime behavior still disagrees.

Why this is distinct from #379

#379 reported that the JSON Schema verifier itself was using Python regex/URI semantics and accepted records that ECMA-262 would reject. #388 corrected that path in validate.py.

This issue is the residual cross-surface divergence after that fix:

# validate.py
pattern.replace("$", r"\Z") \
       .replace(".", r"[^\n\r\u2028\u2029]") \
       .replace(r"\d", "[0-9]")

versus raw Python regexes such as:

# content_marking.py
_SUBJECT_RE = re.compile(r"^(spiffe://[^/]+/.+|did:[a-z0-9]+:.+)$")

and the same form in the sandbox adapter. models.py also carries the raw pattern text.

Observed split already measured in review

During independent review of #390, @lywinged measured current behavior for the same subject rule and found that the model and schema verifier can disagree even though the parity test is green:

trailing character model validate_json
none accepts accepts
U+000A refuses refuses
U+000D accepts refuses
U+2028 accepts refuses

The same review also noted that content_marking.py accepts line-terminator variants through its raw Python regex path while the main verifier rejects the same record.

This is not caused by #390; that review only exposed it.

Why the current parity test can miss it

tests/test_the_schema_and_the_models_agree.py treats identical pattern text as one valid proof of parity.

That is no longer sufficient once one execution path adapts the pattern before evaluation and another does not. The artifacts can be textually identical and behaviorally different.

The invariant we need is therefore behavioral:

for the same field and same semantic rule, every public TRACE surface that validates or constructs that field should agree on the boundary values that matter.

Minimal regression matrix

For the subject rule, exercise at least:

  • valid control;
  • trailing U+000A;
  • trailing U+000D;
  • trailing U+2028;
  • trailing U+2029.

Compare these surfaces:

  1. validate_json;
  2. TrustRecord.model_validate;
  3. content_marking.build_assertion;
  4. sandbox adapter subject validation.

The exact expected result should follow the already-fixed #388 verifier semantics: line terminators do not belong to the accepted subject.

The same pattern should be checked for the digest regex where those helper surfaces compile it independently.

Bounded fix direction

I would avoid another hand-maintained regex rewrite.

Prefer one shared repository-local helper for TRACE's Python execution semantics (or equivalent shared full-match helper), then use it wherever the same schema pattern is interpreted outside validate.py.

At minimum:

  • make the model, content-marking and sandbox paths use the same boundary semantics as validate_json;
  • extend parity coverage from pattern-string equality to behavioral boundary cases;
  • keep this non-normative: the schema/spec rule is already established, this only aligns Python surfaces with it.

Duplicate / ownership check

I searched current issues for U+2028/U+2029, regex parity, content_marking, and model-validation variants.

This also does not overlap #399, which is the separate eat_profile truthiness/type-boundary defect.

AI-assisted source sweep and regression design; @altrudev remains responsible for the contribution.

Activity

  1. github-actions commented on Sep 23, 2026

    @github-actions
    Contributor

    🔴 Contributor Check: HIGH

    Check Result
    Profile HIGH
    Credential LOW
    Overall HIGH

    Automated check by AgenTrust Contributor Check.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    needs-review:HIGHContributor check flagged HIGH risk

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions